Doctoral theses at NTNU, 2010:235 Giancarlo Marafioti ... · NTNU Norwegian University of Science and Technology Thesis for the degree of philosophiae doctor Faculty of Information

Doctoral theses at NTNU, 2010:235

Giancarlo MarafiotiEnhanced Model Predictive Control:Dual Control Approach and StateEstimation Issues

ISBN 978-82-471-2461-1 (printed ver.)ISBN 978-82-471-2462-8 (electronic ver.)

ISSN 1503-8181

NTN

UN

orw

egia

n U

nive

rsity

of

Scie

nce

and

Tech

nolo

gyTh

esis

for

the

degr

ee o

fph

iloso

phia

e do

ctor

Facu

lty

of In

form

atio

n Te

chno

logy

, Mat

hem

atic

s an

dEl

ectr

ical

Eng

inee

ring

Dep

artm

ent o

f Eng

inee

ring

Cyb

erne

ticsD

octoral theses at NTN

U, 2010:235

Giancarlo M

arafioti

NTNUNorwegian University of Science and Technology

Thesis for the degree of philosophiae doctor

Faculty of Information Technology, Mathematicsand Electrical EngineeringDepartment of Engineering Cybernetics

c©Giancarlo Marafioti

ISBN 978-82-471-2461-1 (printed version)ISBN 978-82-471-2462-8 (electronic version)ISSN 1503-8181

ITK Report 2010-15-W

Doctoral Theses at NTNU, 2010:235

Printed by Tapir Uttrykk

























Summary

The main contribution of this thesis is the advancement of Model Predictive Control(MPC). MPC is a well known and widely used advanced control technique, which ismodel-based and capable of handling both input and state/output constraints via recedinghorizon optimization methods. The complex structure of MPC is delineated, and it isshown how improvements of some of its components are able to enhance overall MPCperformance.

In more detail, in Chapter 3, the definition of a state dependent input weight, in thecost function, shows satisfactory controller performance for a large region of workingconditions, compared to a standard MPC formulation. The robustness of the optimizationproblem is improved, i.e., this particular weight configuration yields a ‘better’ conditionedproblem than the standard MPC will produce. This is demonstrated by a simulation studyof a particular Autonomous Underwater Vehicle.

Chapter 4 presents a novel formulation named Persistently Exciting Model Predic-tive Control (PE-MPC). The fundamental idea is that because of its use of a model, MPCshould be amenable to adaptive implementation and on-line tuning of the model. Suchan approach requires guaranteeing certain signal properties, known as ‘persistent excita-tion’, to ensure uniform identifiability of the model. An approach to augment the inputconstraint set of MPC to provide this guarantee is proposed. It is also shown, how thisformulation has properties that are typical of Dual Control problems. Theoretical resultsare proven and simulation-based examples are used to confirm the strength of this newapproach.

Chapter 5 takes state estimation issues into account. Kalman filtering for nonlin-ear systems is described, and throughout the chapter, a comparison of the more recentUnscented Kalman Filter (UKF), with the well known Extended Kalman Filter (EKF) isgiven. Two examples are presented, first a photobioreactor for algae production, and thena simple but effective example of locally weakly unobservable nonlinear system. In bothcases, simulation-based results prove the advantages of using UKF as an alternative toEKF.

i

Summary





i

Summary





i

Summary





i

Summary

ii

Summary

ii

Summary

ii

Summary

ii

Preface

This thesis is submitted in partial fulfillment of the requirements for the degree of Doctorof Philosophy at the Norwegian University of Science and Technology (NTNU).

The research presented is the result of my doctoral studies, mostly carried out at theDepartment of Engineering Cybernetics (NTNU) in the period August 2006 - May 2010.During this time I was privileged to stay at other research institutions. In September 2008- February 2009 I was a scholar at the Department of Mechanical and Aerospace Engi-neering at the University of California San Diego (UCSD). In April 2008, and in April2009, I visited the Department of Automatic Control at the Ecole Supérieur d’Electricité(Supélec) in Gif-sur-Yvette, France.

AcknowledgmentMost of all, I would like to thank my supervisor Professor Morten Hovd, who has givenme the opportunity to take PhD studies in the beautiful city of Trondheim. He has beenof great support and trust during the entire period of my studies. He provided me witha good working environment, and he was always helpful in any circumstances, it was agreat support to know that I could always rely on him. He also introduced me to the worldof academic research, and guided me through the various stages.

I would like also to thank Professor Robert R. Bitmead for taking good care of meduring my stay at UCSD, and especially for guiding me during my research period athis department. With his admirable experience and knowledge he has contributed to myresearch and on improving my skills as researcher and person.

I am grateful to all my coauthors, it has been stimulating to work with all of them.Their remarks, ideas, and all discussions we have had, have contributed to my research. Iwould like also to thank my former office mate Steinar Kolås for sharing his experienceand knowledge with me.

I am glad to have had many interesting colleagues at the departments I have worked/visited. They all have contributed on increasing my knowledge and I have particularlyenjoyed all the discussions during lunch and coffee breaks. In particular I am very happyI shared time and experiences with my colleagues and friends, Francesco Scibilia, Milan

iii

Preface







iii

Preface







iii

Preface







iii

Preface

Milovanovic, Luca Pivano and Ilaria, Roberto Galeazzi, Gullik A. Jensen, Johannes Tjøn-nås, Mernout Burger, Aksel A. Transeth, Esten I. Grøtli, Selvanathan Sivalingam, HardyB. Siahaan. They made my life at the department, and around Norway memorable.

I acknowledge the assistance from Stewart Clark at NTNU for editing this thesis.Many thanks also to the department secretaries, Tove, Unni, Eva, and Bente for their

appreciable help on all the not ‘so friendly’ bureaucracy. Thanks also to Stefano Bertellifor organizing the challenging cage-ball training, it is nice that he cares about the physicalcondition of the department’s employees.

I am grateful to the Research Council of Norway for providing the funding necessaryfor my research.

Finally, I am warmly grateful to my parents, they have always trusted and supportedme, in all my choices, although some decision I made resulted in moving away from them.

Un caloroso ringraziamento alla mia famiglia che ha sempre creduto in me e mi hasupportato in tutte le mie scelte, anche se alcune hanno significato l’allontanarsi da loroper andare a vivere lontano dalla mia terra.

Trondheim, August 2010 Giancarlo Marafioti

iv

Preface








iv

Preface








iv

Preface








iv

Contents

1 Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4 List of publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 An introduction to Model Predictive Control 72.1 Origins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.2 The Receding Horizon Principle . . . . . . . . . . . . . . . . . . . . . . 92.3 Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3.1 Linear Model Predictive Control . . . . . . . . . . . . . . . . . . 122.3.2 Nonlinear Model Predictive Control . . . . . . . . . . . . . . . . 162.3.3 Convex and non-convex Model Predictive Control . . . . . . . . 172.3.4 Explicit Model Predictive Control . . . . . . . . . . . . . . . . . 17

3 Model Predictive Control with state dependent input weight, a simulation-based analysis 193.1 The Autonomous Underwater Vehicle example . . . . . . . . . . . . . . 20

3.1.1 A two-dimensional model . . . . . . . . . . . . . . . . . . . . . 203.2 Model-Based Predictive Control . . . . . . . . . . . . . . . . . . . . . . 24

3.2.1 Internal models used for prediction . . . . . . . . . . . . . . . . 243.2.2 Nonlinear state dependent cost function weight . . . . . . . . . . 253.2.3 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.3 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.4.1 A brief introduction to robustness of MPC controllers . . . . . . . 653.4.2 Industrial approaches to robust MPC . . . . . . . . . . . . . . . . 663.4.3 The state dependent input weight . . . . . . . . . . . . . . . . . 69

3.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

v

Contents










3.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

v

Contents










3.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

v

Contents










3.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

v

Contents

4 Persistently Exciting Model Predictive Control, a dual control approach 734.1 Background results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

4.1.1 Fundamentals of Dual Control theory . . . . . . . . . . . . . . . 764.1.2 Schur complement . . . . . . . . . . . . . . . . . . . . . . . . . 804.1.3 Persistence of excitation for adaptive control schemes . . . . . . . 81

4.2 Persistently Exciting Model Predictive Control . . . . . . . . . . . . . . 844.2.1 Derivation of constraints for persistence of excitation . . . . . . . 844.2.2 Persistently Exciting Model Predictive Control formulation . . . . 87

4.3 Examples using Finite Impulse Response models . . . . . . . . . . . . . 894.3.1 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . 91

4.4 Examples using state space models . . . . . . . . . . . . . . . . . . . . . 1004.4.1 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . 101

4.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

5 State estimation issues 1095.1 The state estimation problem . . . . . . . . . . . . . . . . . . . . . . . . 109

5.1.1 Definitions of Observability . . . . . . . . . . . . . . . . . . . . 1105.2 A solution: Kalman filtering . . . . . . . . . . . . . . . . . . . . . . . . 113

5.2.1 Extended Kalman Filter . . . . . . . . . . . . . . . . . . . . . . 1145.2.2 Unscented Kalman Filter . . . . . . . . . . . . . . . . . . . . . . 115

5.3 Unscented and Extended Kalman filtering for a photobioreactor . . . . . 1195.3.1 Photobioreactor for microalgae production . . . . . . . . . . . . 1205.3.2 Biomass estimation . . . . . . . . . . . . . . . . . . . . . . . . . 124

5.4 State estimation in Nonlinear Model Predictive Control, UKF advantages 1315.4.1 An example of locally weakly unobservable system . . . . . . . . 131

5.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

6 Conclusions and recommendations for further work 1376.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1376.2 Further work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

References 141

vi

Contents






4.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107






5.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135


References 141

vi

Contents






4.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107






5.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135


References 141

vi

Contents






4.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107






5.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135


References 141

vi

List of Tables

1.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

5.1 Model parameters for Porphyridium purpureum at 25 ◦C. . . . . . . . . . 1235.2 Model parameters for [TIC] dynamics. . . . . . . . . . . . . . . . . . . . 1245.3 Mean Squared Error Index . . . . . . . . . . . . . . . . . . . . . . . . . 128

vii

List of Tables

1.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5


vii

List of Tables

1.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5


vii

List of Tables

1.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5


vii

List of Tables

viii

List of Tables

viii

List of Tables

viii

List of Tables

viii

List of Figures

1.1 General MPC structure. . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2.1 Receding Horizon principle. . . . . . . . . . . . . . . . . . . . . . . . . 11

3.1 Underwater vehicle reference frames. . . . . . . . . . . . . . . . . . . . 213.2 AUV real state, and EKF state estimate, using the LTI model and constant

input weight. Initial condition of 20 meters offset from the reference. . . . 313.3 Case with LTI model and constant input weight. Initial condition of 20

meters offset from the reference. . . . . . . . . . . . . . . . . . . . . . . 323.4 AUV real state, and EKF state estimate, using the LTI model and state

dependent input weight. Initial condition of 20 meters offset from thereference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.5 Case with LTI model and state dependent input weight. Initial conditionof 20 meters offset from the reference. . . . . . . . . . . . . . . . . . . . 34

3.6 AUV real state, and its EKF state estimate, using the LTI model and con-stant input weight. Initial condition of 50 meters offset from the reference. 35

3.7 Case with LTI model and constant input weight. Initial condition of 50meters offset from the reference. . . . . . . . . . . . . . . . . . . . . . . 36

3.8 AUV real state, and EKF state estimate, using the LTI model and statedependent input weight. Initial condition of 50 meters offset from thereference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37


3.10 AUV real state, and EKF state estimate, using the LTV model and con-stant input weight. Initial condition of 20 meters offset from the reference. 39

3.11 Case with LTV model and constant input weight. Initial condition of 20meters offset from the reference. . . . . . . . . . . . . . . . . . . . . . . 40

3.12 AUV real state, and EKF state estimate, using the LTV model and statedependent input weight. Initial condition of 20 meters offset from thereference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

ix

List of Figures















ix

List of Figures















ix

List of Figures















ix

List of Figures

3.13 Case with LTV model and state dependent input weight. Initial conditionof 20 meters offset from the reference. . . . . . . . . . . . . . . . . . . . 42

3.14 AUV real state, and its EKF state estimate, using the LTV model and con-stant input weight. Initial condition of 50 meters offset from the reference. 43




3.18 AUV real state, and its EKF state estimate, using the LTV model andconstant input weight. Initial condition of 100 meters offset from thereference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47



3.21 Case with LTV model and state dependent input weight. Initial conditionof 100 meters offset from the reference. . . . . . . . . . . . . . . . . . . 50

3.22 AUV real state, and EKF state estimate, using the LTV-LTI model andconstant input weight. Initial condition of 20 meters offset from the ref-erence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

3.23 Case with LTV-LTI model and constant input weight. Initial condition of20 meters offset from the reference. . . . . . . . . . . . . . . . . . . . . 52

3.24 AUV real state, and EKF state estimate, using the LTV-LTI model andstate dependent input weight. Initial condition of 20 meters offset fromthe reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.25 Case with LTV-LTI model and state dependent input weight. Initial con-dition of 20 meters offset from the reference. . . . . . . . . . . . . . . . 54

3.26 AUV real state, and its EKF state estimate, using the LTV-LTI modeland constant input weight. Initial condition of 50 meters offset from thereference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55



x

List of Figures

















x

List of Figures

















x

List of Figures

















x

List of Figures






3.34 Hessian Condition Numbers, when using the LTV model. Initial condi-tion of 100 meters offset from the reference. . . . . . . . . . . . . . . . . 63

3.35 Hessian Condition Numbers, when using the LTV model. Initial condi-tion of 20 meters offset from the reference. . . . . . . . . . . . . . . . . . 64

4.1 Plant input and output signals. . . . . . . . . . . . . . . . . . . . . . . . 914.2 FIR parameters: RLS estimates (solid line), ‘real’ values (dashed line),

MPC model parameter (stars). . . . . . . . . . . . . . . . . . . . . . . . 924.3 Persistently exciting input and its spectrum indicating excitation suited to

the fitting of a three-parameter model. . . . . . . . . . . . . . . . . . . . 924.4 Plant input and output signals. The dashed line indicates the set-point. . . 944.5 FIR parameters: RLS estimates (solid line), ‘real’ values (dashed line),

MPC model parameter (stars). . . . . . . . . . . . . . . . . . . . . . . . 944.6 Spectrum of the input indicating excitation suited to the fitting of a three-

parameter model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 954.7 Representation of PE-MPC constraints and optimal solution for a short

simulation interval. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 954.8 Plant input and output signals, base example for comparison. The dashed

line indicates the set-point. . . . . . . . . . . . . . . . . . . . . . . . . . 964.9 FIR parameters, base example for comparison: RLS estimates (solid line),

‘real’ values (dashed line), MPC model parameter (stars). . . . . . . . . . 974.10 Plant input and output signals when using longer excitation horizon P .

The dashed line indicates the set-point. . . . . . . . . . . . . . . . . . . . 974.11 FIR parameters when using longer excitation horizon P : RLS estimates

(solid line), ‘real’ values (dashed line), MPC model parameter (stars). . . 984.12 Plant input and output signals when using smaller SRC design parameter

ρ0. The dashed line indicates the set-point. . . . . . . . . . . . . . . . . . 98

xi

List of Figures



















xi

List of Figures



















xi

List of Figures



















xi

List of Figures

4.13 FIR parameters when using smaller SRC design parameter ρ0: RLS esti-mates (solid line), ‘real’ values (dashed line), MPC model parameter (stars). 99

4.14 Input, output and parameters for state space case. . . . . . . . . . . . . . 1024.15 PE-MPC constraints and solved QP problems for state space case. . . . . 1034.16 Input, output and parameters for state space case with Kalman Filter state

estimate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1044.17 PE-MPC constraints and solved QP problems for state space case with

Kalman Filter state estimate. . . . . . . . . . . . . . . . . . . . . . . . . 1054.18 Input and output spectrum, and state and its estimate for state space case

with Kalman Filter state estimate. . . . . . . . . . . . . . . . . . . . . . 106

5.1 Photobioreactor diagram. . . . . . . . . . . . . . . . . . . . . . . . . . 1215.2 UKF estimation for simulated batch mode. . . . . . . . . . . . . . . . . . 1265.3 Experimental data: input and output of the photobioreactor collected in

continuous mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1275.4 Biomass estimation comparison for continuous cultures. . . . . . . . . . 1295.5 UKF parameter estimate and its covariance for continuous culture. . . . . 1305.6 Simulation-based results: EKF as state estimator. . . . . . . . . . . . . . 1335.7 Simulation-based results: UKF as state estimator. . . . . . . . . . . . . . 134

xii

List of Figures








xii

List of Figures








xii

List of Figures








xii

Chapter 1

Introduction

This thesis presents an investigation of Model Predictive Control (MPC) and some relatedissues. MPC has found widespread application in industry, in particular in the chemicalprocessing industries. The main objective of this thesis is to contribute to further enlargethe application area of MPC. In order to put the contributions of the thesis in context,and make it easier for the reader to appreciate these contributions, background materialcovering the fundamental ideas, general classifications and some properties of MPC arepresented. This thesis touches three large branches of control theory, i.e., Model Pre-dictive Control, Dual Control, and State Estimation. Thus it is practically impossible topresent a comprehensive exposition of those control areas of interest. However, in Chap-ter 2, and the first sections in Chapters 3 and 5, background results, general formulations,technical definitions and concepts are given. In addition, the references to the literaturegiven should be suitable as starting points for more in-depth examinations of backgroundconcepts and development.

In the following part of this chapter, the main contributions, thesis structure andassociated publications are indicated.

1.1 MotivationDue to the great success of MPC, see for example Qin & Badgwell [2003], the demand ofapplying this advanced control technique to a larger set of plants is constantly increasing.In Chapter 2, and references therein, some of the most important advantages of this controlapproach are described.

The results obtained in this thesis, are inspired by the desire to extend MPC applica-tions to new areas, and in general to improve its performance. Therefore, an attempt hasbeen made to enhance MPC, by exploiting its structure and properties.

Figure 1.1 presents a general structure of MPC. Clearly, there are several parts (shownas blocks) which are interconnected (as illustrated by the arrows), resulting in a complex

1

Chapter 1

Introduction






1

Chapter 1

Introduction






1

Chapter 1

Introduction






1

1. Introduction

Plant

Stateestimator

Cost function& constraints

Optimizer

Model

MPC controllerReference

Disturbances

Figure 1.1: General MPC structure.

structure. In general, it is desired to control a specific plant by manipulating its inputsand using its outputs to gather state information. However, an external disturbance mayaffect plant operations. A controller, based on a model of the plant, tries to optimize aspecific control cost function, while satisfying a set of constraints and following a givenreference. The model is generally unable to exactly represent the plant dynamics, thus areceding horizon is introduced, and the control function is re-optimized at every time step,as means to obtain some feedback. This is necessary to reduce the effects of model/plantmismatch, and also to counteract the effect of unknown disturbances. Also, it is likelythat the measured plant output does not contain all information needed for the controller,thus a state estimator is often used to overcome this problem.

To enhance MPC performance, and extend its range of application, focus is mainlyplaced on the ‘cost function & constraints’, ‘model’, and ‘state estimator’ blocks of Figure1.1. More in detail, the results obtained in Chapter 3, enhance the MPC cost function toincrease the region of operation where the controller has satisfactory performance. Thisis desired, and in the past gain-scheduling techniques were developed to obtain similarresults. The particular nonlinear input weight introduced, that depends on the state ofthe system, has also benefits for the implicit robustness of the problem formulation withrespect to numerical solutions. Finally, the chapter illustrates the use of different modelrepresentations in the MPC formulation.

In Chapter 4 the constraint set is properly augmented to incorporate dual controlfeatures. This allows the particular MPC to work in an adaptive context, where ‘learning’algorithms may be implemented to estimate, and update the MPC model. Finally, in

2

1. Introduction

Plant

Stateestimator


Optimizer

Model


Disturbances





2

1. Introduction

Plant

Stateestimator


Optimizer

Model


Disturbances





2

1. Introduction

Plant

Stateestimator


Optimizer

Model


Disturbances





2

Contribution

Chapter 5, the focus is on state estimation, and it is shown how improving this part of thesystem has direct impact on the proper control operation. If the estimate has poor quality,the controller may be not able to achieve its task.

1.2 ContributionThe main contributions in this thesis are:

• The state dependent input weight introduction for MPC applied to an AutonomousUnderwater Vehicle (AUV). The standard MPC formulation is enhanced by theintroduction of a state dependent cost function input weight. This has similarityto gain-scheduling controllers, where control of nonlinear systems is obtained byseveral sub-controllers, designed to provide satisfactory performance, each one in asub-set of operating conditions. The AUV example also results in an ill-conditionedoptimization problem. The introduction of the novel input weight has benefits onthe Hessian condition number, improving robustness for optimization algorithms.

• The formulation of Persistently Exciting Model Predictive Control (PE-MPC), todeal with parameter estimation in an adaptive context. In general, when adaptionis required in closed-loop systems, persistence of excitation becomes an issue dueto the conflict between the control and adaption actions. PE-MPC includes bothactions in its formulation, yielding a control signal that is obtained as a trade-offbetween these two considerations. This is a feature that is present in dual controlproblems.

• Methods are studied for state estimations in nonlinear systems. The commonly usedExtended Kalman Filter is compared to the more recently developed UnscentedKalman Filter. The first comparison scenario is the implementation of both filterin a photobioreactor that produces microalgae. The operational data are availableand are used to validate the results. The second comparison scenario consists of asimulation study of a locally weakly unobservable system. In both examples theUKF is found to give superior performance.

3

Contribution






3

Contribution






3

Contribution






3

1. Introduction

1.3 Thesis Organization

The remainder of the thesis is organized as follows:

• Chapter 2 - A general introduction to Model Predictive Control is presented. A his-torical discussion is given about its origins and the receding horizon principle. Forlinear state space models, and for Finite Impulse Response (FIR) models, standardMPC formulations, and their equivalent Quadratic Programming problems are pre-sented. Thereafter, there is a description of a general Nonlinear MPC formulation(NMPC), and the classification into Convex or Non-convex optimization problems.Explicit MPC is briefly introduced. These formulations are important because areused in the subsequent chapters to present the main contributions.

• Chapter 3 - A novel MPC formulation with state dependent input weight in the costfunction is given. This is analyzed in a simulation study controlling an AutonomousUnderwater Vehicle (AUV). Its nonlinear model description is presented. Severallinear model approximations are obtained. With respect to the AUV example a statedependent input weight is defined, and its application benefits are shown. Combin-ing different model approximations, the standard MPC, and the particular inputweight, six different controllers are obtained. Their performance are compared bysimulation, and finally discussed from a robust MPC point of view.

• Chapter 4 - A novel MPC formulation, named Persistently Exciting Model Predic-tive Control, is presented. The MPC problem is discussed in an adaptive context,where data quality requirements for model adjustment are built into the MPC for-mulation. The relationship to the dual control problem is explained. Results ondual control theory, and the Persistence of Excitation Condition (PEC) are consid-ered fundamental for understanding the context of the new formulation. Thereforethey are given in a background section. The proposed approach augments the inputconstraint set of MPC to guarantee a sufficient excitation level such that adaptiveestimation algorithms can be used to estimate and update the model uncertain pa-rameters. Finally, an example using FIR models and the Recursive Least Squarealgorithm for parameter estimation is presented.

• Chapter 5 - The state estimation problem is discussed. This mainly focuses onKalman filtering for nonlinear systems. Two popular nonlinear estimators, the Ex-tended Kalman Filter (EKF) and the Unscented Kalman Filter (UKF) are comparedin two different examples. The first performance comparison is presented by es-timating the state of a photobioreactor for microalgae production. The obtainedresults are then validated using plant operational data. The second filter perfor-

4

1. Introduction







4

1. Introduction







4

1. Introduction







4

List of publications

mance comparison is obtained in an NMPC scenario defined by a nonlinear systemwith observability issues. Simulation-based examples show the advantage of usingUKF over EKF.

• Chapter 6 - General conclusions, and recommendations for further work are given.

1.3.1 NotationIn this thesis, conventional notation for control theory is used. For instance, scalars willbe denoted by plain symbols, vectors by lowercase bold symbols, matrices by uppercasebold symbols, vector/matrix transpose by (·)T , positive definiteness by (·) > 0 or negativedefiniteness by (·) < 0, the n-dimensional identity matrix, by In. Through the exposi-tion of results, when new notation is introduced, then it is properly defined. For ease ofconsultation, some of the most used notation is listed in Table 1.1.

Table 1.1: NotationSymbol Representations, S Scalar, plain characterv,M Vector or matrix, bold character(·)T Vector or matrix transpose

(·) > 0 Positive definite for matrices, positive for scalars(·) < 0 Negative definite for matrices, negative for scalarsIn Identity matrix of (n× n) dimensions

1.4 List of publicationsMost of the material contained in this thesis has been either published or recently submit-ted for publication. The following is the list of publications related to the work presentedin this thesis.

Book chapter• [Marafioti et al., 2009a] - G. Marafioti, S. Olaru, M. Hovd. State Estimation in

Nonlinear Model Predictive Control, Unscented Kalman Filter Advantages. InNonlinear Model Predictive Control Towards New Challenging Applications Se-ries: Lecture Notes in Control and Information Sciences, Vol. 384 - L. Magni, D.M. Raimondo, F. Allgöwer (Eds.), 2009 - ISBN: 978-3-642-01093-4.

5










5










5










5

1. Introduction

Journal papers• [Marafioti et al., 2010a] - G. Marafioti, R.R. Bitmead, M. Hovd. Persistently Ex-

citing Model Predictive Control. Submitted to International Journal of AdaptiveControl and Signal Processing.

Conference papers• [Marafioti et al., 2010b] - G. Marafioti, R.R. Bitmead, M. Hovd. Persistently Ex-

citing Model Predictive Control using FIR models. In Proceedings of Cyberneticsand Informatics International Conference, Vyšná Boca, Slovak Republic, February10-13, 2010.

• [Marafioti et al., 2009b] - G. Marafioti, S. Tebbani, D. Beauvois, G. Becerra, A.Isambert, M. Hovd. Unscented Kalman Filter state and parameter estimation in aphotobioreactor for microalgae production. In Proceedings of International Sympo-sium on Advanced Control of Chemical Processes, Koç University, Istanbul, Turkey,July 12-15, 2009.

• [Marafioti et al., 2008b] - G. Marafioti, S. Olaru, M. Hovd. State Estimation inNonlinear Model Predictive Control, Unscented Kalman Filter Advantages. In Pro-ceedings of the International Workshop on Assessment and Future Directions ofNonlinear Model Predictive Control, Pavia, Italy, September 5-9, 2008.

• [Marafioti et al., 2008a] - G. Marafioti, R.R. Bitmead, M. Hovd. Model PredictiveControl with State Dependent Input Weight: an Application to Underwater Vehi-cles. In Proceedings of the 17th IFAC World Congress, Seoul, Korea, July 6-11,2008.

The results, either published or submitted, are partially or fully related to the thesis chap-ters as follow.

• Chapter 3: Marafioti et al. [2008a]

• Chapter 4: Marafioti et al. [2010a,b]

• Chapter 5: Marafioti et al. [2008b, 2009a,b]

6

1. Introduction












6

1. Introduction












6

1. Introduction












6

Chapter 2

An introduction to Model PredictiveControl

Model Predictive Control (MPC), also referred as receding horizon control or movinghorizon control, is an advanced control technique. In more detail, it is an optimal con-trol procedure that easily allows engineers to enter constraints directly into the controlproblem formulation. Furthermore, it explicitly uses a process model to predict the futureresponse of a plant. In general, MPC is applied in a discrete time framework, althoughin the literature continuous time formulations are available. At each control interval,the future plant behavior is optimized according to some cost criterium, and an associ-ated constraint set. As result of the optimization, a sequence of manipulated variables isobtained. Thus, the first element of this sequence is sent into the plant, and the entirecomputation is repeated at subsequent control intervals.

MPC-based technology was originally developed for controlling petroleum refiner-ies and power plants. Nowadays, it is possible to find this technology in a wide rangeof applications, such as chemical industries, food processing, automotive, and aerospaceindustries. The development of MPC techniques is due to the increasing research effortfrom both the academic community and industry. Thus, by understanding MPC proper-ties, it is possible to improve performance, to add new features, and expand applicability.

The importance of taking constraints into account arises from the reason that mostof real world systems are subject to physical constraints. Thus, the ability of handlingconstraints in a straightforward way makes MPC very attractive. For instance, this isparticularly helpful in process industries where the most profitable operations are oftenobtained when running at one or more constraints. Constraints are often associated withavailable range of actuators, safe operating regions, product quality specifications. Forexample, when manufacturing a product that requires heat, being able to minimize theenergy needed for heating can reduce the production cost, however a certain amount ofheat is needed to guarantee that the final product is correctly manufactured. This defines a

7

Chapter 2





7

Chapter 2





7

Chapter 2





7

2. An introduction to Model Predictive Control

quality specification that the addition of heat is intended to ensure. Therefore, this may berepresented by a constraint. For a general plant, there may be input constraints (saturationon actuators), state and/or output constraints (safety limits on pressure or temperatures),and they may be represented as inequality or equality and then included into MPC formu-lation.

Another interesting feature of MPC is the adoption of a model to predict the fu-ture plant behavior. These models are obtained from simple plant tests, or by applyingsome system identification technique to the plant operational data. However, the effortof designing better plant models, the increasing availability of computational power, theimprovement of optimization algorithms, the constant pursuit of better performance, allcontribute to increasing the trend of using first-principle models. In general these are non-linear models obtained directly from the application of well established laws of physicsand chemistry. For instance, in a chemical plant they can be derived starting from thechemical transformations and reactions occurring inside a process.

In a fairly recent survey on industrial MPC technology (Qin & Badgwell [2003]),the authors claim that there are thousands of successful applications, and this number isstill increasing. An interesting introduction, with a complete mathematical formulation,is given in Findeisen & Allgöwer [2002], where theoretical, computational, and imple-mentation aspects are discussed. A less mathematically formal introduction, but compre-hensive and straightforward to understand is Camacho & Bordons [2007]. Several bookshave been written about linear MPC. Some of these are Maciejowski [2002], Rossiter[2003], Camacho & Bordons [2004]. They describe and analyze MPC theory and its ap-plication in slightly different ways. In Maciejowski [2002] more effort is spent on howto handle constraints, and the author starts from a simple formulation adding step by stepmore details such as integral effect and constraint softening. In Rossiter [2003] MPC isviewed from a practical approach, avoiding jargon, and giving interesting examples aboutMPC tuning and numerical issues. Finally, in Camacho & Bordons [2004] an extensivereview of MPC controllers is given, with focus on a variant of MPC known as Gener-alized Predictive Control. The authors attempt to reduce the distance between the waypractitioners use control algorithms and the more abstract way researchers formulate thecontrol technique.

A more recent book, and perhaps the one that can be considered the most complete,is Rawlings & Mayne [2009]. There, it is finally possible to find a comprehensive andfundamental analysis of MPC theory and design, with an in-depth examination and com-parison of the most used, and recent, state estimation techniques.

This chapter gives a general MPC description. A brief historical introduction is pre-sented first. Then starting with the well known linear quadratic regulation problem, theconcept of a receding horizon is introduced as way to introduce feedback. Finally, generalMPC formulations are given, and used in the remaining part of the thesis, where severalproblems of relevance to MPC are addressed. It is assumed that that the inputs are con-

8







8







8







8

Origins

tinuously valued (in space). Moreover, the focus in this chapter is on quadratic objectivefunctions and linear constraints, because this particular MPC is predominant in industryand it is related to the LQR problem.

2.1 Origins

The control action of MPC is determined as the result of an online optimization problem.As consequence, the behavior of an MPC controller may be rather complex. The earliestMPC implementations were limited by the low computational resource availability. Al-though only steady-state models were used, depending on the plant dimension, severalhours could be needed for the optimization problem solution.

The first ideas on receding horizon control and model predictive control can be tracedback to the 1960s [García et al., 1989]. In Propoi [1963] the receding horizon approachwas proposed. However, real interest in this innovative approach started later, in the1980s, after the first publications on Model Algorithmic Control (MAC) [Richalet et al.,1978], Dynamic Matrix Control (DMC) [Cutler & Ramaker, 1979, 1980], and a completedescription on Generalized Predictive Control (GPC) [Clarke et al., 1987a,b].

DMC was developed mostly for the oil and chemical industries. Its goal was to tacklemultivariable constrained control problems. In fact, the previous approach to these prob-lems, was to solve them by integrating several single loop controllers augmented by manytime-delay compensators, overrides, selectors, etc. With DMC, a time domain model,either finite impulse or step response, was used to predict the plant behavior. The origi-nal DMC formulation was completely deterministic with no explicit disturbance model.GPC formulation was designed as a new adaptive control scheme, where transfer func-tion models were adopted. Since it does not come naturally to use GPC with multivariableconstrained systems, DMC was adopted by most of the oil and chemical companies.

As mentioned in Qin & Badgwell [2003], the success of DMC in the oil and chemicalindustries attracted the academic community. Therefore, primordial research on MPC wasintended as an attempt to understand DMC.

2.2 The Receding Horizon Principle

Before discussing the receding horizon principle, let us briefly introduce the well knownLinear Quadratic Regulation (LQR) problem for discrete systems, [Naidu, 2003]. Simply

9

Origins


2.1 Origins







9

Origins


2.1 Origins







9

Origins


2.1 Origins







9


stated, the LQR problem is

minx,u

N∑i=0

xTi Qxi + uT

i Rui (2.1a)

xi+1 = Axi +Bui, given x0, (2.1b)

where the optimization horizon N may be either finite or infinite. It is possible to writethe optimal solution of (2.1) in the feedback form

uopti = Kix

opti , i = 1, . . . , N (2.2)

where the state feedback gain Ki is calculated from the corresponding Riccati equation.For example, for an infinite horizon LQR the optimal state feedback gain is

K∞ = − (R+BTPB

)−1BTPA, (2.3)

where P is the unique positive definite solution to the discrete time algebraic Riccatiequation

P = Q+AT(P − PB

(R+BTPB

)−1BTP

)A. (2.4)

Now, if we add inequality constraints to (2.1), it is not possible to find a closed-formsolution such as (2.2). However, online optimization, and a finite length horizon can be astrategy to overcome this problem.

The idea is to re-formulate the constrained, infinite horizon, optimization problem asa constrained, finite horizon, optimization problem. This may be done by representing the‘tail’ of the infinite horizon as a ‘cost to go’ term to be added to the existing cost function.

Note that this re-formulation results in an open-loop optimization, therefore if modelerrors and/or disturbances occur it is necessary to introduce some feedback techniqueto compensate for both. The function of receding (or moving) horizon is to introducefeedback. This finite horizon, which shifts as the time passes, is the basic idea of anyMPC. Figure 2.1 illustrates this principle. Given the last measurement available at thepresent time k, the controller predicts the dynamic behavior of plant, over a predictionhorizon Np. This is based on an internal model of the system. To introduce feedbackonly the first element of the predicted optimal input sequence is applied to the plant untilthe next measurement is available. Then, horizons are shifted one step forward, and anew optimization problem is formulated and solved. Note in Figure 2.1, the differencebetween the set-point and the reference trajectory. A set-point is a constant reference thatthe output should ideally follow. Instead, a reference trajectory defines how the plantshould reach the set-point from the current output.

10



minx,u

N∑i=0

xTi Qxi + uT

i Rui (2.1a)



uopti = Kix

opti , i = 1, . . . , N (2.2)


K∞ = − (R+BTPB

)−1BTPA, (2.3)


P = Q+AT(P − PB

(R+BTPB

)−1BTP

)A. (2.4)




10



minx,u

N∑i=0

xTi Qxi + uT

i Rui (2.1a)



uopti = Kix

opti , i = 1, . . . , N (2.2)


K∞ = − (R+BTPB

)−1BTPA, (2.3)


P = Q+AT(P − PB

(R+BTPB

)−1BTP

)A. (2.4)




10



minx,u

N∑i=0

xTi Qxi + uT

i Rui (2.1a)



uopti = Kix

opti , i = 1, . . . , N (2.2)


K∞ = − (R+BTPB

)−1BTPA, (2.3)


P = Q+AT(P − PB

(R+BTPB

)−1BTP

)A. (2.4)




10

The Receding Horizon Principle

k k + Np

PresentPast Future

Closed-loopinputs

Open-loop inputs

Open-loop outputs

Set-point

Control horizonk + Nc

Prediction horizon

Closed-loopoutputs

Reference

Future closed-loop input

Figure 2.1: Receding Horizon principle.

11


k k + Np

PresentPast Future

Closed-loopinputs

Open-loop inputs

Open-loop outputs

Set-point


Prediction horizon

Closed-loopoutputs

Reference



11


k k + Np

PresentPast Future

Closed-loopinputs

Open-loop inputs

Open-loop outputs

Set-point


Prediction horizon

Closed-loopoutputs

Reference



11


k k + Np

PresentPast Future

Closed-loopinputs

Open-loop inputs

Open-loop outputs

Set-point


Prediction horizon

Closed-loopoutputs

Reference



11


Some considerations are needed regarding the model used for prediction. As men-tioned for example in Bemporad [2006], it is necessary to observe that models are ob-tained as a tradeoff between their accuracy and the complexity of resultant optimizations.In fact, while models used for simulations tend to have the best accuracy to numericallyreproduce the behavior with the minimum error, prediction models are usually more sim-ple, but yet they are able to capture the main dynamics of the system.

Nowadays, an essential MPC model distinction must be made between linear andnonlinear models. The application of the first types result in linear MPC formulations,while the second types produce Nonlinear MPC (NMPC) formulations. In general, non-linear models are more complex and the associated optimization problem solution is com-putationally demanding. Moreover, the most important intention of this classification isthat in general when a nonlinear model is used the optimization loses its convexity prop-erty. Thus, NMPC is a non-convex problem, while MPC is a convex one. The implicationsof this distinction are discussed briefly in a succeeding section in this chapter.

In the following section, a detailed MPC formulation is presented and its propertiesare given.

2.3 Formulations

2.3.1 Linear Model Predictive Control

There are several linear MPC formulations in the literature. In this section, two verycommon discrete time formulations are described. The first one uses state space models,while the second one uses finite impulse response models for prediction.

Linear MPC is used in process control even if the process to control is nonlinear.The reason is that when the process is working in the neighborhood of some equilibriumpoint, a linear model is often a good approximation of its dynamics. In such applications,MPC is used primarily to reject disturbances, while optimizing the operation of the plantin some sense.

State space models

Consider the following discrete state space model

xk+1 = Axk +Buk (2.5)

12





2.3 Formulations




State space models


xk+1 = Axk +Buk (2.5)

12





2.3 Formulations




State space models


xk+1 = Axk +Buk (2.5)

12





2.3 Formulations




State space models


xk+1 = Axk +Buk (2.5)

12

Formulations

for prediction of the open-loop system, where x is the state vector, u is the input vector,and k is the time step index. Using (2.5) the MPC finite-time optimal control problem is

minu

J(k, u) = xTk+Np

Sxk+Np+

Np−1∑j=0

xTk+jQxk+j + uk

k+jTRuk

k+j (2.6a)

s.t. xk+1+j = Axk+j +Bukk+j , from xk, j = 0, . . . , Np − 1

(2.6b)

umin ≤ ukk+j ≤ umax , j = 0, . . . , Np − 1

(2.6c)

xmin ≤ xk+j ≤ xmax , j = 1, . . . , Np

(2.6d)

lmin ≤ L(xk+j) ≤ lmax , j = 1, . . . , Np

(2.6e)

where Np is the prediction horizon, uk = [ukkT,uk

k+1T, . . . ,uk

k+Np−1T]T is the sequence

of manipulate variables or inputs, S = ST ≥ 0, Q = QT ≥ 0, R = RT > 0 are weightmatrices, of appropriate dimensions, defining the performance index, and umin, umax,xmin, and xmax are vectors of appropriate dimensions defining input, state, constraints,respectively. Finally, L(x) and lmin, lmax indicate the possibility to have constraints onlinear combinations of states.

The state at time k can be written as a function of the initial state x0 and the previousinputs uk−1−i (i = 0, 1, . . . , k − 1)

xk = Ax0 +k−1∑i=0

AiBuk−1−i. (2.7)

Thus, now (2.6) can be reformulated as the following QP problem

u∗(x0) = argminu

1

2uTHu+ xT

0GT u+

1

2xT0Y x0 (2.8a)

s.t. Lu ≤ W + V x0, (2.8b)

where

u∗(x0) =[uT∗

0 , . . . ,uT∗Np−1

]T(2.9)

is the optimal solution, and H = HT ≥ 0, G, Y , L, W , and V are matrices of ap-propriate dimensions (see Bemporad et al. [2004]). Note that the term 1

2xT0Y x0 can be

dropped since it does not affect the optimization result. Simply, the difference is that the

13

Formulations


minu

J(k, u) = xTk+Np

Sxk+Np+

Np−1∑j=0

xTk+jQxk+j + uk

k+jTRuk

k+j (2.6a)


(2.6b)


(2.6c)

xmin ≤ xk+j ≤ xmax , j = 1, . . . , Np

(2.6d)


(2.6e)


k+1T, . . . ,uk




xk = Ax0 +k−1∑i=0

AiBuk−1−i. (2.7)


u∗(x0) = argminu

1

2uTHu+ xT

0GT u+

1

2xT0Y x0 (2.8a)

s.t. Lu ≤ W + V x0, (2.8b)

where

u∗(x0) =[uT∗

0 , . . . ,uT∗Np−1

]T(2.9)


2xT0Y x0 can be


13

Formulations


minu

J(k, u) = xTk+Np

Sxk+Np+

Np−1∑j=0

xTk+jQxk+j + uk

k+jTRuk

k+j (2.6a)


(2.6b)


(2.6c)

xmin ≤ xk+j ≤ xmax , j = 1, . . . , Np

(2.6d)


(2.6e)


k+1T, . . . ,uk




xk = Ax0 +k−1∑i=0

AiBuk−1−i. (2.7)


u∗(x0) = argminu

1

2uTHu+ xT

0GT u+

1

2xT0Y x0 (2.8a)

s.t. Lu ≤ W + V x0, (2.8b)

where

u∗(x0) =[uT∗

0 , . . . ,uT∗Np−1

]T(2.9)


2xT0Y x0 can be


13

Formulations


minu

J(k, u) = xTk+Np

Sxk+Np+

Np−1∑j=0

xTk+jQxk+j + uk

k+jTRuk

k+j (2.6a)


(2.6b)


(2.6c)

xmin ≤ xk+j ≤ xmax , j = 1, . . . , Np

(2.6d)


(2.6e)


k+1T, . . . ,uk




xk = Ax0 +k−1∑i=0

AiBuk−1−i. (2.7)


u∗(x0) = argminu

1

2uTHu+ xT

0GT u+

1

2xT0Y x0 (2.8a)

s.t. Lu ≤ W + V x0, (2.8b)

where

u∗(x0) =[uT∗

0 , . . . ,uT∗Np−1

]T(2.9)


2xT0Y x0 can be


13


QP cost function value will be different from the one in (2.8a), but in MPC, but the valueof u at the optimal solution will be identical.

At every iteration k, the MPC algorithm consists of measuring or estimating the statex0, then solving the QP problem (2.8), and finally applying only the first element of theoptimal sequence (2.9), uk = u∗

0(x0), to the process. Then the procedure is repeated forthe subsequent time step k + 1.

It is possible to extend (2.6) in several ways. For example, in order to obtain atracking formulation such that an output vector yk = Cxk follows a certain referencesignal rk, the cost function (2.6a) may be replaced by

Np−1∑j=0

(yk+j − rk+j)TQy(yk+j − rk+j) + Δuk

k+jTRΔuk

k+j, (2.10)

where Qy = QTy ≥ 0 is a matrix of output weights, and Δuk

k � ukk − uk

k−1 is the newoptimization variable, possibly constrained as Δumin ≤ Δuk

k ≤ Δumax. As for the QPformulation, it is sufficient to define the tracking vector

[xTk , r

Tk , u

kk−1

T]T and use it in

the same way in (2.7).If it is necessary to reduce the optimization complexity, an MPC with multiple hori-

zons may be used. It is possible to have a prediction horizon NP , and a control horizonNc ≤ Np. A shorter control horizon reduces the number of optimization variables, yield-ing a smaller optimization problem. Similarly, forcing the input(s) to be constant overseveral time steps (so-called ’input blocking’) also reduces the size of the optimizationproblem.

A problem that may occur with MPC is that the optimization problem is infeasible.The feasibility of MPC is connected with constraint satisfaction. That is, a constraint setdefines the space where candidate solutions (feasible solutions) may be searched to findthe optimal solution with respect to the cost function, and outside the constraint set thereexists no solution satisfying all constraints. An MPC may become infeasible for severalreasons. For instance, a large disturbance may occur and as consequence it may be notpossible to keep the plant within the specified constraints. Moreover, it is also difficult toanticipate when an MPC becomes infeasible, thus a strategy to recover from infeasibilityis essential for practical implementations.

Constraint softening may be used as an expedient to deal with infeasibility. Forexample, to implement output constraint softening, it is necessary to modify the outputconstraint as follows

ymin − ευmin ≤ yk+j ≤ ymax + ευmax (2.11)

where ε is a slack variable introduced to allow constraint violation, υmin and υmax arevectors with nonnegative elements. There are also terms (linear and/or quadratic) that

14






Np−1∑j=0


k+jTRΔuk

k+j, (2.10)


k � ukk − uk



[xTk , r

Tk , u

kk−1

T]T and use it in







14






Np−1∑j=0


k+jTRΔuk

k+j, (2.10)


k � ukk − uk



[xTk , r

Tk , u

kk−1

T]T and use it in







14






Np−1∑j=0


k+jTRΔuk

k+j, (2.10)


k � ukk − uk



[xTk , r

Tk , u

kk−1

T]T and use it in







14

Formulations

are added to the cost function in order to penalize the constraint violation. Typically,these terms are designed to ensure that any constraint violation is kept small. The ad-ditional terms in the cost function, together with the magnitude of the elements in vmin,vmax, determine how soft the corresponding constraint will be. Note that MPC with softconstraints as (2.11) can be still formulated as a QP problem.

Other techniques for dealing with infeasibility are to remove the constraints at thebeginning of the horizon [Rawlings & Muske, 1993]. When input constraints representa physical limitation, only state constraints can be softened. However, not all constraintsmay be relaxed at the same way, and generally softening one constraint instead of anothermay have less effect on the overall MPC performance. To prioritize which constraint hasto be softened in an optimal fashion, it is possible to apply more complex and effectivemethods as in Scokaert & Rawlings [1999] or in Vada et al. [1999].

FIR models

It is well known that stable processes can be represented by Finite Impulse Response(FIR) models. For a Single-Input Single-Output (SISO) system, the FIR model is

yk = φkθT + vk, (2.12)

where {vk} is a Gaussian measurement noise sequence, θT =[θ1 θ2 . . . θn

]is the

vector of impulse coefficients, the regressor vector is φk =[uk−1 uk−2 . . . uk−n

]Twith {uk−i}ni=1 past n inputs to the process, finally n is the number of Markov parameters.

The FIR-based MPC may be formulated as

minuk

J(k, u) =1

2

Np−1∑j=0

‖ yk+j+1 ‖2Q + ‖ ukk+j ‖2R, (2.13a)

s.t. yk+j = φk+jθTk , j = 1, . . . , Np, (2.13b)

umin ≤ ukk+j ≤ umax, j = 0, . . . , Np − 1, (2.13c)

ymin ≤ yk+j ≤ ymax, j = 1, . . . , Np, (2.13d)

where uk =[ukk uk

k+1 . . . ukk+Np−1

]Tis the control sequence vector, and Q and R are

the cost function output and input weights, respectively.Clearly, the solution of (2.13) may be found by converting the problem into an equiv-

alent Quadratic Programming (QP) problem. QP problems (with positive definite Hessianmatrices) are known to be convex, and thus have a unique unique global optimum. Ef-ficient and reliable algorithms are available to solve them. Straightforward manipulation

15

Formulations



FIR models


yk = φkθT + vk, (2.12)


]is the




minuk

J(k, u) =1

2

Np−1∑j=0

‖ yk+j+1 ‖2Q + ‖ ukk+j ‖2R, (2.13a)




where uk =[ukk uk

k+1 . . . ukk+Np−1




15

Formulations



FIR models


yk = φkθT + vk, (2.12)


]is the




minuk

J(k, u) =1

2

Np−1∑j=0

‖ yk+j+1 ‖2Q + ‖ ukk+j ‖2R, (2.13a)




where uk =[ukk uk

k+1 . . . ukk+Np−1




15

Formulations



FIR models


yk = φkθT + vk, (2.12)


]is the




minuk

J(k, u) =1

2

Np−1∑j=0

‖ yk+j+1 ‖2Q + ‖ ukk+j ‖2R, (2.13a)




where uk =[ukk uk

k+1 . . . ukk+Np−1




15


yield the standard QP formulation

minU

1

2U ′HU + gTU (2.14a)

s.t. Umin ≤ U ≤ Umax (2.14b)

where H is the Hessian matrix, and g the gradient vector.

2.3.2 Nonlinear Model Predictive ControlAssume that

xk+1 = f(xk,uk,ωk;θ) (2.15a)yk = h(xk,uk,vk;θ) (2.15b)

is the plant to be controlled, where xk ∈ Rn, uk ∈ Rm, yk ∈ Rp, ωk and vk aresequences of independent zero-mean random variables, and θ ∈ RN denotes the vectorof unknown plant parameters.

Let uk = [ukkT,uk

k+1T, . . . ,uk

k+Np−1T]T be the control sequence vector where Np is

the prediction horizon assumed coincident with the control horizon for simplicity. Thestandard Model Predictive Control is specified by solving the following finite horizonoptimal control problem posed at time k from plant state value xk, see Mayne [2000],and using plant model parameter estimate θ.

minuk

J(k, u) = F (xk+Np) +

Np−1∑j=0

g(xk+j,ukk+j) (2.16a)

s.t. xk+j+1 = f(xk+j,ukk+j, θ), from xk, j = 1, . . . , Np − 1,

(2.16b)

ukk+j ∈ U , j = 0, . . . , Np − 1,

(2.16c)

xk+j ∈ X , j = 0, . . . , Np, (2.16d)

where the following conditions are assumed to hold:• the dynamics f(·), the terminal cost F (·), and the stage cost g(·) are continuous;• f(0, 0) = 0, F (0) = 0, g(0, 0) = 0;• the process noise in (2.16b) is replaced by its (zero) mean, and the parameter vectorθ is considered constant along the horizon Np;

• U ⊆ Rm contains the origin in its interior;• X ⊆ Rn contains the origin in its interior.

16



minU

1

2U ′HU + gTU (2.14a)






Let uk = [ukkT,uk

k+1T, . . . ,uk



minuk

J(k, u) = F (xk+Np) +

Np−1∑j=0



(2.16b)

ukk+j ∈ U , j = 0, . . . , Np − 1,

(2.16c)

xk+j ∈ X , j = 0, . . . , Np, (2.16d)



16



minU

1

2U ′HU + gTU (2.14a)






Let uk = [ukkT,uk

k+1T, . . . ,uk



minuk

J(k, u) = F (xk+Np) +

Np−1∑j=0



(2.16b)

ukk+j ∈ U , j = 0, . . . , Np − 1,

(2.16c)

xk+j ∈ X , j = 0, . . . , Np, (2.16d)



16



minU

1

2U ′HU + gTU (2.14a)






Let uk = [ukkT,uk

k+1T, . . . ,uk



minuk

J(k, u) = F (xk+Np) +

Np−1∑j=0



(2.16b)

ukk+j ∈ U , j = 0, . . . , Np − 1,

(2.16c)

xk+j ∈ X , j = 0, . . . , Np, (2.16d)



16

Formulations

2.3.3 Convex and non-convex Model Predictive Control

The concept of convexity is very important in optimization. Problems that are convex aregenerally easier to solve. Convex optimization is a very well defined and studied area[Boyd & Vandenberghe, 2004]. Convex problems have the characteristic that if a localsolution exists, it is also a global solution. Standard Quadratic Programming optimizationproblems (QPs) are a subset of the class of convex optimization problems. QPs haveconvex objective functions, linear equality constraints and linear inequality constraints.

MPC with a linear model, a convex cost function, and a convex set of constraints,results in a convex optimization problem. As seen previously, formulations (2.6) and(2.13) can be easily converted to standard QP problems. For solving these problems,efficient and reliable algorithms are available [Nocedal & Wright, 2000].

Nonlinear models in MPC lead to non-convex optimization problems. A method forsolving non-convex optimization problems is Sequential Quadratic Programming (SQP)[Nocedal & Wright, 2000]. SQP is one of the most popular methods for solving nonlin-early constrained optimization problems. Fundamentally, an SQP algorithm decomposesthe non-convex problem into a series of QPs, and sequentially solves them. The maindrawback of SQP algorithms is their computational complexity, especially if comparedwith QP algorithms. Furthermore, SQP will typically find a locally optimal solution, butthis local solution is not guaranteed to also be the globally optimal solution.

The next chapter describes an example of MPC, where a novel state dependent weightis defined. Despite the fact that the model is nonlinear, several approaches using linearizedapproximated models and a standard QP solver are used, instead of SQP solvers.

2.3.4 Explicit Model Predictive Control

When an MPC is implemented, it is well known that at every sampling time an opti-mization problem must be solved online. Although faster algorithms are developed andcomputational power availability increases constantly, for some systems with fast dynam-ics it is not possible to apply the standard MPC. Explicit MPC has been developed as asolution to this problem. The basic idea is to transfer the computation offline. This is doneby computing a closed-form solution to the optimization problem, leading to a reductionin the complexity of the online algorithm.

An algorithm to explicitly determine the linear quadratic state feedback control lawfor constrained linear systems is developed in Bemporad et al. [2002]. It is shown thatthe control law obtained is piece-wise linear and continuous, and the only online compu-tation consists of a simple evaluation of an explicitly defined piece-wise linear function.Another interesting algorithm is developed in Tøndel et al. [2003], where the results fromBemporad et al. [2002] are extended avoiding unnecessary partitioning, and improvingthe efficiency in the online evaluation.

17

Formulations









17

Formulations









17

Formulations









17


One should note, however, that Explicit MPC algorithms are applicable in practiceonly to systems with few states (not much higher than five). This is caused mainly bythe fast increase of the table dimensions, required to define the piece-wise linear func-tions. The offline computation for obtaining the explicit solution, and the organizationfor an efficient online evaluation, becomes prohibitive for systems with a large number ofstates. Present research is focused on sub-optimal Explicit MPC in order to find a simplersolution that can be applicable to larger systems. Extensions to nonlinear systems areinvestigated as well.

For the reader who wishes to find a more detailed introduction of this approach, astarting point can be Kvasnica [2009].

18




18




18




18

Chapter 3

Model Predictive Control with statedependent input weight, asimulation-based analysis

This chapter presents a simulation-based study that investigates and compares the effectsof different cost function formulations, as well as several model simplifications using lin-earization, for application of linear(ized) MPC to a nonlinear system. An AutonomousUnderwater Vehicle (AUV) is used for the study. The main contribution of this chapteris to introduce and illustrate the effects of a novel state dependent input weight, whichis shown to improve performance compared to other simple MPC formulation modifica-tions.

A nonlinear two-dimensional model is described, and it is used as the plant. Twodifferent linearized models are implemented in the MPC, a Linear Time Invariant (LTI)and a Linear Time Varying (LTV), respectively. Controlling a nonlinear system by usinglinear models is common in MPC practice. MPC implementation for this particular AUVsystem is interesting, especially because it is possible to appreciate the significant controlperformance improvement when a nonlinear input weight for the MPC cost function iscoupled with the LTV model.

For this application, the state dependent input weight idea in the cost function is anovelty, and resembles the use of a varying controller gain adopted in gain schedulingtechniques [Rugh & Shamma, 2000]. In fact, due to the different input weights the con-troller gain, obtained at every time step after solving a QP optimization problem, variesas a function of AUV position. The state dependent input weight improves the solutionrobustness of QP problem.

The LTI model yields a standard linear MPC, which has the least computationaldemand. The LTV model yields an MPC which is still convex, but since the model statespace matrices are computed every sampling time, and they are different for each time step

19

Chapter 3






19

Chapter 3






19

Chapter 3






19

3. Model Predictive Control with state dependent input weight, a simulation-basedanalysis

within the prediction horizon, the overall problem requires more computational effort.However, its solution requires less effort than an NMPC problem.

The underwater vehicle dynamics of the AUV are described in Sutton & Bitmead[1998]. It moves in a two-dimensional space, and it has a constant forward speed (surge).Thus, by controlling the vehicle rudder angle, that is physically constrained between[−0.5, 0.5] radians, the goal is to follow the ocean floor at a certain constant distance.A rigorous mathematical formulation is given in next section.

3.1 The Autonomous Underwater Vehicle exampleTo illustrate the benefits of the state dependent input weight described in detail in Section3.2.2 a simulation-based analysis of different approaches for controlling an AutonomousUnderwater Vehicle (AUV) is presented.

The particular AUV model chosen has some features that make it of relevant interest.That is, due to the limited range of achievable control values the vehicle is unable to ac-celerate in an arbitrary direction. The response of the AUV to positive step changes in therudder turning is initially to accelerate in the opposite direction, this behavior is reminis-cent of non-minimum phase systems. These observations would suggest not attemptingthe use of feedback control which relies on high-gain or on system inversion. Finally, thisclass of systems may lead to difficulties in addressing issues on robustness and tuning ofthe controller. For more details see Santos & Bitmead [1995].

Note that the choice of taking this AUV example is made in order to illustrate theresults using a relatively simple model, which still has the capability to display complexnonlinear behavior. The purpose of this chapter is not to make a realistic contributionto the control of underwater vehicles, which would clearly require a model in a three-dimensional space. However, this example makes clear the contribution of the state de-pendent input weight, highlighting the benefits.

For more details on underwater vehicles see Fossen [1994], which gives a compre-hensive and extensive discussion on the guidance and control of ocean vehicles.

3.1.1 A two-dimensional modelFigure 3.1 shows a sketch of the vehicle. To describe the AUV dynamics a model isdefined with respect to two frames of reference. This gives the possibility to know theposition and orientation of the vehicle while moving in its environment. The vehicledynamics, in continuous time, is given in the body-fixed frame defined by Cxz. Thesurge and heave speed are defined by u1 and w, respectively. The pitch angle rate is

1To have a standard description of the underwater vehicle model, an abuse of notation is required. Inthis case, u denotes the state variable surge, and not the system input.

20










20










20










20

The Autonomous Underwater Vehicle example

Figure 3.1: Underwater vehicle reference frames.

denoted q. Thus (u, w, q) are respectively, the forward, perpendicular and anti-clockwiserotational velocities of the submarine along, to and around its major axis in the body-fixedframe.

In the earth-fixed reference frame OXZ we describe the motion of the vehicle asshown in (3.2). The attitude of the vehicle is defined by θc. The vehicle dynamics isdescribed, according to Sutton & Bitmead [1998], as

MI

⎡⎣ u(t)w(t)q(t)

⎤⎦ = mq(t)

⎡⎣0 −1 01 0 00 0 0

⎤⎦⎡⎣u(t)w(t)q(t)

⎤⎦+ u(t)Dh

⎡⎣u(t)w(t)q(t)

⎤⎦+ γg(t) + ucw(t) + ucp(t)

(3.1)

⎡⎣xc(t)zc(t)

θc(t)

⎤⎦ =

⎡⎣ cos(θc(t)) sin(θc(t)) 0− sin(θc(t)) cos(θc(t)) 0

0 0 1

⎤⎦⎡⎣u(t)w(t)q(t)

⎤⎦ (3.2)

where MI is the inertia matrix including the hydrodynamic added mass, m is the vehiclemass, Dh is the damping matrix, and the buoyancy term γg(t) is zero because the vehicleis assumed to be neutrally buoyant. ucw(t) and ucp(t) are respectively the forces and themoments generated by the rudder and propeller. Their expressions are given by (3.3-3.12).

21





MI

⎡⎣ u(t)w(t)q(t)

⎤⎦ = mq(t)

⎡⎣0 −1 01 0 00 0 0

⎤⎦⎡⎣u(t)w(t)q(t)

⎤⎦+ u(t)Dh

⎡⎣u(t)w(t)q(t)


(3.1)

⎡⎣xc(t)zc(t)

θc(t)

⎤⎦ =


0 0 1

⎤⎦⎡⎣u(t)w(t)q(t)

⎤⎦ (3.2)


21





MI

⎡⎣ u(t)w(t)q(t)

⎤⎦ = mq(t)

⎡⎣0 −1 01 0 00 0 0

⎤⎦⎡⎣u(t)w(t)q(t)

⎤⎦+ u(t)Dh

⎡⎣u(t)w(t)q(t)


(3.1)

⎡⎣xc(t)zc(t)

θc(t)

⎤⎦ =


0 0 1

⎤⎦⎡⎣u(t)w(t)q(t)

⎤⎦ (3.2)


21





MI

⎡⎣ u(t)w(t)q(t)

⎤⎦ = mq(t)

⎡⎣0 −1 01 0 00 0 0

⎤⎦⎡⎣u(t)w(t)q(t)

⎤⎦+ u(t)Dh

⎡⎣u(t)w(t)q(t)


(3.1)

⎡⎣xc(t)zc(t)

θc(t)

⎤⎦ =


0 0 1

⎤⎦⎡⎣u(t)w(t)q(t)

⎤⎦ (3.2)


21


v2 � u2 + w2, (3.3)

ε � sin−1(wv

), (3.4)

Jp �u

|νDp| , (3.5)

where ε is the angle between the Cx axis and the velocity vector v, which is not neces-sarily the same as θc. Jp is the propeller advancement coefficient, ν is the propeller shaftrotational speed, and Dp is the propeller diameter.

Cxsw � −Cxow − cwC2

zow(ε+ β)2

2πbw, (3.6)

Czsw � −Czow(ε+ β)− 2.1(ε+ β)2, (3.7)

ucw = [ucw11 , ucw21 , ucw31 ]T , (3.8)

ucw11 = 0.5ρSwv2(Czsw sin(β) + Cxsw cos(β)), (3.9)

ucw21 = 0.5ρSwv2(Czsw cos(β)− Cxsw sin(β)), (3.10)

ucw31 = −ucw21(0.2cw cos(β) + daw)− ucw11(0.2cw sin(β)), (3.11)

ucp =

⎡⎣ρ|ν|νD4

p(Ct0p + Ct1pJp + Ct2pJ2p + Ct3pJ

3p )

−ρ|ν|D3pwCn

ρ|ν|D3pwCnDap

⎤⎦ , (3.12)

where Sw = bwcw is the rudder surface, and ρ is the sea water density. Cxow, Czow, daw,bw, cw, are the rudder characteristics, and Ct0p, Ct1p, Ct2p, Ct3p, Cn, Dap, are the propellercharacteristics. More details can be found in Santos & Bitmead [1995].

The vehicle has two inputs that can be used for control purposes. They are the rudderdeflection β, and the propeller rotation frequency ν, respectively. The ocean bottom po-sition is modeled as a disturbance to be rejected by the controller. Its model is describedby state xf which is the rate of change of the absolute angle of the ocean bottom definedby θ∗(t) at x∗

f (t). The angular velocity of the sea floor θ∗(t) is modeled as the negativeoutput of a first order filtered white noise process driven by ξ(t)

xf (t) = Afxf (t) + Bfξ(t) (3.13a)f(t) = Cfxf (t) (3.13b)

The relative angle θ between the ocean bottom and the vehicle is given by

θ(t) = θc(t)− θ∗(t), (3.14)

22


v2 � u2 + w2, (3.3)

ε � sin−1(wv

), (3.4)

Jp �u

|νDp| , (3.5)



zow(ε+ β)2

2πbw, (3.6)

Czsw � −Czow(ε+ β)− 2.1(ε+ β)2, (3.7)

ucw = [ucw11 , ucw21 , ucw31 ]T , (3.8)




ucp =

⎡⎣ρ|ν|νD4


3p )

−ρ|ν|D3pwCn

ρ|ν|D3pwCnDap

⎤⎦ , (3.12)






θ(t) = θc(t)− θ∗(t), (3.14)

22


v2 � u2 + w2, (3.3)

ε � sin−1(wv

), (3.4)

Jp �u

|νDp| , (3.5)



zow(ε+ β)2

2πbw, (3.6)

Czsw � −Czow(ε+ β)− 2.1(ε+ β)2, (3.7)

ucw = [ucw11 , ucw21 , ucw31 ]T , (3.8)




ucp =

⎡⎣ρ|ν|νD4


3p )

−ρ|ν|D3pwCn

ρ|ν|D3pwCnDap

⎤⎦ , (3.12)






θ(t) = θc(t)− θ∗(t), (3.14)

22


v2 � u2 + w2, (3.3)

ε � sin−1(wv

), (3.4)

Jp �u

|νDp| , (3.5)



zow(ε+ β)2

2πbw, (3.6)

Czsw � −Czow(ε+ β)− 2.1(ε+ β)2, (3.7)

ucw = [ucw11 , ucw21 , ucw31 ]T , (3.8)




ucp =

⎡⎣ρ|ν|νD4


3p )

−ρ|ν|D3pwCn

ρ|ν|D3pwCnDap

⎤⎦ , (3.12)






θ(t) = θc(t)− θ∗(t), (3.14)

22


which gives

θ(t) = θc(t)− θ∗(t)

= q(t) + Cfxf (t) (3.15)

where θ∗(t) = −f(t) = −Cfxf (t).The relative distance between the ocean bottom and the vehicle center is given by

κ(t) and its rate of change is computed with

κ(t) = u(t) sin(θ(t))− w(t) cos(θ(t)). (3.16)

It is assumed that the distance between the vehicle center and the ocean bottom is mea-sured, and the measurement is affected by a white Gaussian noise η

y(t) = κ(t) + η(t). (3.17)

For representing the system (3.1-3.17) in a more compact form, let us define the statevector

x(t) =

⎡⎢⎢⎢⎢⎢⎢⎣

uwqθκxf

⎤⎥⎥⎥⎥⎥⎥⎦, (3.18)

thus (3.1-3.17) becomes

x(t) = f (x(t), β(t), ξ(t)) (3.19a)y(t) = Cx(t) + η(t) (3.19b)

where the dynamics (3.19a) is nonlinear, and the measurement (3.19b) is linear, with

C =

⎡⎢⎢⎢⎢⎢⎢⎣

000010

⎤⎥⎥⎥⎥⎥⎥⎦

T

. (3.20)

The vehicle parameters can be found in Santos & Bitmead [1995]. The system (3.19)has non-minimum phase behavior, in the sense that a positive input step change on βmakes the vehicle accelerate in the opposite direction before converging eventually to thesteady state value of the response. Furthermore, an inability to accelerate in an arbitrarydirection, due to the limited range of the control value reachable ucw, makes the sys-tem nonholonomic. These considerations make the control of the vehicle interesting andsuitable for investigating model predictive control features.

23


which gives

θ(t) = θc(t)− θ∗(t)

= q(t) + Cfxf (t) (3.15)





y(t) = κ(t) + η(t). (3.17)


x(t) =

⎡⎢⎢⎢⎢⎢⎢⎣

uwqθκxf

⎤⎥⎥⎥⎥⎥⎥⎦, (3.18)




C =

⎡⎢⎢⎢⎢⎢⎢⎣

000010

⎤⎥⎥⎥⎥⎥⎥⎦

T

. (3.20)


23


which gives

θ(t) = θc(t)− θ∗(t)

= q(t) + Cfxf (t) (3.15)





y(t) = κ(t) + η(t). (3.17)


x(t) =

⎡⎢⎢⎢⎢⎢⎢⎣

uwqθκxf

⎤⎥⎥⎥⎥⎥⎥⎦, (3.18)




C =

⎡⎢⎢⎢⎢⎢⎢⎣

000010

⎤⎥⎥⎥⎥⎥⎥⎦

T

. (3.20)


23


which gives

θ(t) = θc(t)− θ∗(t)

= q(t) + Cfxf (t) (3.15)





y(t) = κ(t) + η(t). (3.17)


x(t) =

⎡⎢⎢⎢⎢⎢⎢⎣

uwqθκxf

⎤⎥⎥⎥⎥⎥⎥⎦, (3.18)




C =

⎡⎢⎢⎢⎢⎢⎢⎣

000010

⎤⎥⎥⎥⎥⎥⎥⎦

T

. (3.20)


23


3.2 Model-Based Predictive ControlThe standard linear MPC framework (2.6) is implemented. However, since the AUVmodel is nonlinear, (3.19) is linearized. Two different strategies are applied, using first anLTI internal model, and then an LTV model.

3.2.1 Internal models used for predictionThe internal linear model is obtained by linearizing the underwater vehicle dynamics(3.19). Thus, for a sufficiently small interval around the equilibrium point, it is possibleto suitably approximate the nonlinear behavior of the AUV by the state space system

x(t) = Ax(t) +Bβ(t) (3.21a)y(t) = Cx(t) (3.21b)

where A and B are the following Jacobians

A =∂f(x, β)

∂x

∣∣∣∣x=x∗,β=β∗

(3.22)

B =∂f(x, β)

∂β

∣∣∣∣x=x∗,β=β∗

(3.23)

calculated about some point (x∗, β∗) in the combined state and input space. The measure-ment matrix C is given in (3.20).

In MPC using linearized models, is it common practice to linearize around an equi-librium point, and to ‘move the origin to the equilibrium point’ by the introduction ofdeviation variables. Here, the AUV is moving forward along some trajectory, and thedeviation variables in (3.21) are therefore relative to the nominal trajectory.

The MPC linear model thus obtained is sufficiently accurate when AUV is close tothe point where the linearization is obtained. However, when increasing the initial offsetbetween the working point and the initial condition the LTI approach fails. Therefore, asa candidate strategy for solving this issue, an LTV model is implemented as follow.

x(t) = Atx(t) +Btβ(t) (3.24a)y(t) = Ctx(t) (3.24b)

where At and Bt are computed by

At =∂f(x, β)

∂x

∣∣∣∣x=x∗(t),β=β∗(t)

(3.25)

Bt =∂f(x, β)

∂β

∣∣∣∣x=x∗(t),β=β∗(t)

(3.26)

24






A =∂f(x, β)

∂x

∣∣∣∣x=x∗,β=β∗

(3.22)

B =∂f(x, β)

∂β

∣∣∣∣x=x∗,β=β∗

(3.23)






At =∂f(x, β)

∂x

∣∣∣∣x=x∗(t),β=β∗(t)

(3.25)

Bt =∂f(x, β)

∂β

∣∣∣∣x=x∗(t),β=β∗(t)

(3.26)

24






A =∂f(x, β)

∂x

∣∣∣∣x=x∗,β=β∗

(3.22)

B =∂f(x, β)

∂β

∣∣∣∣x=x∗,β=β∗

(3.23)






At =∂f(x, β)

∂x

∣∣∣∣x=x∗(t),β=β∗(t)

(3.25)

Bt =∂f(x, β)

∂β

∣∣∣∣x=x∗(t),β=β∗(t)

(3.26)

24






A =∂f(x, β)

∂x

∣∣∣∣x=x∗,β=β∗

(3.22)

B =∂f(x, β)

∂β

∣∣∣∣x=x∗,β=β∗

(3.23)






At =∂f(x, β)

∂x

∣∣∣∣x=x∗(t),β=β∗(t)

(3.25)

Bt =∂f(x, β)

∂β

∣∣∣∣x=x∗(t),β=β∗(t)

(3.26)

24

Model-Based Predictive Control

and Ct is (3.20). Note that β∗(t) and x∗(t) define the trajectory where the Jacobians arecomputed, and are generated by using the MPC optimal solution at the previous step, asthe algorithm in Section 3.2.3 describes.

Since MPC requires a discrete time framework, (3.21) is discretized by using thestandard Euler approximation

x(t) � xk+1 − xk

δ(3.27)

where δ is the sampling time.Analogously, for LTV model we have that

Ak+i = I + δAt, Bk+i = δBt, (3.28)

where k is the current discrete time step, and i = {0, 1, . . . , Np − 1} indicates the stepinterval where the discretized Jacobian is applied, within the MPC prediction horizon.

Although the LTV approach allows the AUV to meet the control goal, the transientresponse of the system is not satisfactory, as discussed more in detail in the simulationsin Section 3.3. Thus a nonlinear state dependent cost function is defined as shown next.

3.2.2 Nonlinear state dependent cost function weightIn Sutton & Bitmead [1998] the following cost function is defined

J =

Np−1∑i=0

y2i +Rcβ2i +Qcθ

2i (3.29)

where Rc is the input weight, and Qc is a penalty on the angle θ in order to avoid thevehicle heading backward.

To use (3.29) as the MPC objective function, it is then necessary to reformulate it ina more standard form, resulting in

J(x, β) = (xNp − xref )TS(xNp − xref )

+

Np−1∑i=0

(xi − xref )TQ(xi − xref ) + (βi − βref )R(βi − βref ) (3.30)

where

S = Q =

⎡⎢⎢⎢⎢⎣0 0 0 0 00 0 0 0 00 0 Qc 0 00 0 0 1 00 0 0 0 0

⎤⎥⎥⎥⎥⎦ , R = Rc, (3.31)

25




x(t) � xk+1 − xk

δ(3.27)


Ak+i = I + δAt, Bk+i = δBt, (3.28)




J =

Np−1∑i=0

y2i +Rcβ2i +Qcθ

2i (3.29)




+

Np−1∑i=0


where

S = Q =

⎡⎢⎢⎢⎢⎣0 0 0 0 00 0 0 0 00 0 Qc 0 00 0 0 1 00 0 0 0 0

⎤⎥⎥⎥⎥⎦ , R = Rc, (3.31)

25




x(t) � xk+1 − xk

δ(3.27)


Ak+i = I + δAt, Bk+i = δBt, (3.28)




J =

Np−1∑i=0

y2i +Rcβ2i +Qcθ

2i (3.29)




+

Np−1∑i=0


where

S = Q =

⎡⎢⎢⎢⎢⎣0 0 0 0 00 0 0 0 00 0 Qc 0 00 0 0 1 00 0 0 0 0

⎤⎥⎥⎥⎥⎦ , R = Rc, (3.31)

25




x(t) � xk+1 − xk

δ(3.27)


Ak+i = I + δAt, Bk+i = δBt, (3.28)




J =

Np−1∑i=0

y2i +Rcβ2i +Qcθ

2i (3.29)




+

Np−1∑i=0


where

S = Q =

⎡⎢⎢⎢⎢⎣0 0 0 0 00 0 0 0 00 0 Qc 0 00 0 0 1 00 0 0 0 0

⎤⎥⎥⎥⎥⎦ , R = Rc, (3.31)

25


and

xi =

⎡⎢⎢⎢⎢⎢⎢⎣

ui

wi

qiθiκi

xfi

⎤⎥⎥⎥⎥⎥⎥⎦, xref =

⎡⎢⎢⎢⎢⎢⎢⎣

000000

⎤⎥⎥⎥⎥⎥⎥⎦, βref = 0 (3.32)

To improve control performance, a computationally effective solution is to define thenonlinear input weight

Rc = Rc(k) = aκ2k + b (3.33)

where κk is the relative depth, between the reference (that is a constant distance from theocean floor) and the AUV center of gravity, at the time step k, a and b are positive constantdesign parameters. Thus, for every time step a new input weight is calculated.

From (3.19b) it is possible to note that κ is the only state directly measured. Asgeneral practice, when implementing MPC, a full state knowledge is needed. Thus, anExtended Kalman Filter (EKF) is implemented for estimating the AUV state. In particular,for calculating the nonlinear weight the estimate κ is used instead. With this framework,the measurement noise is also filtered by the EKF. Thus, the input weight is computed by

Rc(k) = aκ2k + b. (3.34)

Finally, the use of (3.34) improves optimization solution robustness as discussed inthe end of this chapter.

3.2.3 AlgorithmsThe discrete MPC algorithm is applied to the underwater vehicle system (3.19). For thelinear MPC formulation with the LTI model, the MPC described in Chapter 2 is applied.In this section, we go through the LTV-MPC algorithm details. Thus, the tail of theoptimal solution at the previous time step k − 1 is defined as

β∗k−1,Np

= [β∗k|k−1, β

∗k+1|k−1, . . . , β

∗k+Np−1|k−1, 0]

T , (3.35)

which is used for computing the trajectory x∗k−1,Np

about where the Jacobians (3.28) willbe calculated.

LTV-MPC algorithm

Consider a generic sampling time instant k, then the LTV-MPC algorithm is definedby the following steps:

26


and

xi =

⎡⎢⎢⎢⎢⎢⎢⎣

ui

wi

qiθiκi

xfi

⎤⎥⎥⎥⎥⎥⎥⎦, xref =

⎡⎢⎢⎢⎢⎢⎢⎣

000000

⎤⎥⎥⎥⎥⎥⎥⎦, βref = 0 (3.32)


Rc = Rc(k) = aκ2k + b (3.33)



Rc(k) = aκ2k + b. (3.34)



β∗k−1,Np

= [β∗k|k−1, β

∗k+1|k−1, . . . , β

∗k+Np−1|k−1, 0]

T , (3.35)



LTV-MPC algorithm


26


and

xi =

⎡⎢⎢⎢⎢⎢⎢⎣

ui

wi

qiθiκi

xfi

⎤⎥⎥⎥⎥⎥⎥⎦, xref =

⎡⎢⎢⎢⎢⎢⎢⎣

000000

⎤⎥⎥⎥⎥⎥⎥⎦, βref = 0 (3.32)


Rc = Rc(k) = aκ2k + b (3.33)



Rc(k) = aκ2k + b. (3.34)



β∗k−1,Np

= [β∗k|k−1, β

∗k+1|k−1, . . . , β

∗k+Np−1|k−1, 0]

T , (3.35)



LTV-MPC algorithm


26


and

xi =

⎡⎢⎢⎢⎢⎢⎢⎣

ui

wi

qiθiκi

xfi

⎤⎥⎥⎥⎥⎥⎥⎦, xref =

⎡⎢⎢⎢⎢⎢⎢⎣

000000

⎤⎥⎥⎥⎥⎥⎥⎦, βref = 0 (3.32)


Rc = Rc(k) = aκ2k + b (3.33)



Rc(k) = aκ2k + b. (3.34)



β∗k−1,Np

= [β∗k|k−1, β

∗k+1|k−1, . . . , β

∗k+Np−1|k−1, 0]

T , (3.35)



LTV-MPC algorithm


26

Simulation results

1. use βk−1, yk−1 and EKF to estimate the state xk−1;2. use the nonlinear model (3.19) and the tail of the optimal solution at the previous

step (3.35) to compute the trajectory

x∗k−1,Np

= [x∗k|k−1,x

∗k+1|k−1, . . . ,x

∗k+Np−1|k−1,x

∗k+Np|k−1]

T ; (3.36)

3. compute the Jacobians (3.28), for i = 0, 1, . . . , Np − 1, about the predicted x∗k−1,Np

and β∗k−1,Np

;4. cast the MPC problem as a QP and then find its optimal solution;5. from the optimal solution

β∗k = [β∗

k|k, β∗k+1|k, . . . , β

∗k+Np−1|k]

T , (3.37)

apply β∗k|k to the underwater vehicle, increment k and go to the algorithm step 1.

Moreover, when the state dependent weight is applied either to an LTI model ap-proach, or to LTV-MPC, it is sufficient to include an extra step (between step 3 and step4), which consist on calculating the state dependent nonlinear weight (3.34).

3.3 Simulation resultsA series of MATLAB simulations are carried out to analyze and compare control perfor-mance of the different frameworks presented. The AUV is released some distance abovethe desired depth relative to the sea floor, and should quickly descend to the desired depthand then follow the sea floor at this depth. Note that in the following, the surge u isassumed constant and not estimated. This is a reasonable assumption since the task ofbottom following will typically require a nominal constant value u = u0. Furthermore,since this is primarily the concern of the rotational shaft speed ν control, the regulation ofu can be separated from the regulation of the other variables, which focuses on the use ofrudder angle β to permit bottom following.

The following corresponding MPC controllers are defined. In detail, their character-istics are:

(C1) - LTI internal model with constant input weight in the MPC cost function;(C2) - LTI internal model with state dependent input weight in the MPC cost function;(C3) - LTV internal model with constant input weight in the MPC cost function;(C4) - LTV internal model with state dependent input weight in the MPC cost function.(C5) - LTV-LTI internal model with constant input weight in the MPC cost function;(C6) - LTV-LTI internal model with state dependent input weight in the MPC cost func-

tion.

27

Simulation results



x∗k−1,Np

= [x∗k|k−1,x

∗k+1|k−1, . . . ,x

∗k+Np−1|k−1,x

∗k+Np|k−1]

T ; (3.36)


and β∗k−1,Np


β∗k = [β∗

k|k, β∗k+1|k, . . . , β

∗k+Np−1|k]

T , (3.37)






tion.

27

Simulation results



x∗k−1,Np

= [x∗k|k−1,x

∗k+1|k−1, . . . ,x

∗k+Np−1|k−1,x

∗k+Np|k−1]

T ; (3.36)


and β∗k−1,Np


β∗k = [β∗

k|k, β∗k+1|k, . . . , β

∗k+Np−1|k]

T , (3.37)






tion.

27

Simulation results



x∗k−1,Np

= [x∗k|k−1,x

∗k+1|k−1, . . . ,x

∗k+Np−1|k−1,x

∗k+Np|k−1]

T ; (3.36)


and β∗k−1,Np


β∗k = [β∗

k|k, β∗k+1|k, . . . , β

∗k+Np−1|k]

T , (3.37)






tion.

27


Note that LTV-LTI is an MPC where the model used for prediction is linear time invariantwithin the prediction horizon, however it is re-linearized about the current state every timestep, hence having a time-varying behavior.

All six controllers are re-formulated as QP problems, which are solved online, withthe standard MATLAB QP solver ‘quadprog’. Relevant simulation parameters are: sam-pling period δ = 0.2 s; input constraints |β| � 0.5 rad; cost function state weightQc = 10, and input weight Rc = 150, when controllers (C1), (C3), and (C5) are used;when controllers (C2), (C4), and (C6) are implemented, the input weight is the one de-fined in (3.34), with a = 1 and b = 150, while the state weight is unchanged.

In all simulation instances, the same ocean bottom profile is considered. This is doneby using the same noise sequence ξ in (3.13a). Finally, the common control goal is tofollow the ocean bottom with a 10 m offset.

Simulation-based analysis

• In Figures 3.2, 3.4, 3.6, 3.8, 3.10, 3.12, 3.14, 3.16, 3.18, 3.20, 3.22, 3.24, 3.26, 3.28,3.30, 3.32 θv = −π/2 indicates the vertical trajectory, that would be the shortestpath for the AUV to follow for reaching the desired depth.

• Figures 3.2 and 3.3 are obtained by applying the MPC approach (C1). Startingfrom an initial offset of 20 m, the controller is able to drive the AUV to the desiredaltitude (10 m distance from the ocean bottom) and to follow the seafloor profile.Note that in the first time steps the input constraint is hit (Figure 3.3(a)), moreoversince the system has non-minimum phase, note how the vehicle starts going on theopposite direction for eventually converging on the desired path (Figure 3.3(b)). Asvisible in Figure 3.3(a), the EKF state estimation gives very satisfactory estimates.

• Figures 3.4 and 3.5 are obtained by applying the MPC approach (C2). These resultsare very similar to the previous analysis. Thus, when starting from a 20 m offset,the controller with the nonlinear input weight has the same performance as the onewith the constant weight.

• Figures 3.6 and 3.7 are obtained by applying the MPC approach (C1). Starting froman initial offset of 50 m, the controller is not able to regulate the system. This is dueto the poor approximation from the LTI model.

• Figures 3.8 and 3.9 are obtained by applying the MPC approach (C2). Startingfrom 50 m offset, the variable input weight is not a sufficient improvement and thecontroller fails as in the previous case.Thus to improve the prediction accuracy, the LTV model is used instead with thefollowing results.

• Figures 3.10 and 3.11 are obtained by applying the MPC approach (C3). Startingfrom an initial offset of 20 m, the controller performance is very similar to the LTImodel approach.

28












28












28












28

Simulation results

• Figures 3.12 and 3.13 are obtained by applying the MPC approach (C4). Again,from an initial offset of 20 m, the introduction of the state dependent input weightyield similar results to the LTI model approach with and without variable inputweight, and also similar to the LTV approach.

• Figures 3.14 and 3.15 are obtained by applying the MPC approach (C3). This time,although starting from an initial offset of 50 m, the controller is able to make theAUV following the ocean bottom. However, as shown in Figure 3.15(a), in thetransient part, the input presents large oscillations that could damage the actuatorsystem. Thus, even if the control goal is reached, this solution it is not practicable.

• Figures 3.16 and 3.17 are obtained by applying the MPC approach (C4). Startingfrom 50 m offset, this time, due to the gain scheduling property of the variable inputweight and to the better approximation of the LTV model, the AUV is able to followthe ocean bottom, without any potentially dangerous transient.

• Figures 3.18 and 3.19 are obtained by applying the MPC approach (C3). Startingfrom 100 m offset, and using only the LTV model with a constant input weight thecontrol goal is met, but again with a not satisfactory input transient.

• Figures 3.20 and 3.21 are obtained by applying the MPC approach (C4). Also from100 m offset, the combined approach is able to produce a very smooth control inputfollowing the ocean bottom.

Approaches (C5) and (C6) allow some interesting observations to be made. In fact,as shown in:

• Figures 3.22, 3.23, obtained by implementing the MPC (C5), starting from 20 moffset;






all controllers manage to follow the ocean bottom. This is due to better accuracy of time-varying models. However, it is possible to note the presence of input oscillations, that arelarger when starting from a larger offset. These oscillations are due to the poor accuracyof the model within the prediction horizon.

29

Simulation results














29

Simulation results














29

Simulation results














29


Clearly, the best approach is to use the LTV model with the state dependent inputweight. Here it is interesting to note how the variable weight in the cost function im-proves the Hessian condition number as explained in Section 3.2.2. Figures 3.34, and3.35 show how switching from constant to state dependent input weight, reduces the Hes-sian condition number, yielding a well-conditioned optimization problem.

30



30



30



30

Simulation results

0 10 20 30 40 50 602.5

3

3.5State (red), State Estimate (blue)

u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]

Figure 3.2: AUV real state, and EKF state estimate, using the LTI model and constantinput weight. Initial condition of 20 meters offset from the reference.

31

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


31

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


31

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


31


0 10 20 30 40 50 60

0

Control Input

Time [s]

0 10 20 30 40 50 60

0

5

10

15

20

25

Time [s]

(a) AUV Control Input and Measured Output.

60 80

(b) AUV trajectory in the earth-fixed frame.

Figure 3.3: Case with LTI model and constant input weight. Initial condition of 20 metersoffset from the reference.

32


0 10 20 30 40 50 60

0

Control Input

Time [s]

0 10 20 30 40 50 60

0

5

10

15

20

25

Time [s]


60 80



32


0 10 20 30 40 50 60

0

Control Input

Time [s]

0 10 20 30 40 50 60

0

5

10

15

20

25

Time [s]


60 80



32


0 10 20 30 40 50 60

0

Control Input

Time [s]

0 10 20 30 40 50 60

0

5

10

15

20

25

Time [s]


60 80



32

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]

Figure 3.4: AUV real state, and EKF state estimate, using the LTI model and state depen-dent input weight. Initial condition of 20 meters offset from the reference.

33

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


33

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


33

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


33


0 10 20 30 40 50 60

0

Control Input

Time [s]

[rad

]

0 10 20 30 40 50 60

0

5

10

15

20

25Measured Output

Time [s]


0 20 40 60 80 100 120 140 160

0


Figure 3.5: Case with LTI model and state dependent input weight. Initial condition of 20meters offset from the reference.

34


0 10 20 30 40 50 60

0

Control Input

Time [s]

[rad

]

0 10 20 30 40 50 60

0

5

10

15

20

25Measured Output

Time [s]


0 20 40 60 80 100 120 140 160

0



34


0 10 20 30 40 50 60

0

Control Input

Time [s]

[rad

]

0 10 20 30 40 50 60

0

5

10

15

20

25Measured Output

Time [s]


0 20 40 60 80 100 120 140 160

0



34


0 10 20 30 40 50 60

0

Control Input

Time [s]

[rad

]

0 10 20 30 40 50 60

0

5

10

15

20

25Measured Output

Time [s]


0 20 40 60 80 100 120 140 160

0



34

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

10

v / 2

0 10 20 30 40 50 6040

50

60

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]

Figure 3.6: AUV real state, and its EKF state estimate, using the LTI model and constantinput weight. Initial condition of 50 meters offset from the reference.

35

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

10

v / 2

0 10 20 30 40 50 6040

50

60

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


35

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

10

v / 2

0 10 20 30 40 50 6040

50

60

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


35

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

10

v / 2

0 10 20 30 40 50 6040

50

60

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


35


0 10 20 30 40 50 60

0

Control Input

Time [s]

0 10 20 30 40 50 6042

44

46

48

50

52Measured Output

Time [s]




36


0 10 20 30 40 50 60

0

Control Input

Time [s]

0 10 20 30 40 50 6042

44

46

48

50

52Measured Output

Time [s]




36


0 10 20 30 40 50 60

0

Control Input

Time [s]

0 10 20 30 40 50 6042

44

46

48

50

52Measured Output

Time [s]




36


0 10 20 30 40 50 60

0

Control Input

Time [s]

0 10 20 30 40 50 6042

44

46

48

50

52Measured Output

Time [s]




36

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

10

v / 2

0 10 20 30 40 50 6040

60

80

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


37

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

10

v / 2

0 10 20 30 40 50 6040

60

80

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


37

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

10

v / 2

0 10 20 30 40 50 6040

60

80

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


37

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

10

v / 2

0 10 20 30 40 50 6040

60

80

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


37


0 10 20 30 40 50 60

0

0 10 20 30 40 50 6040

45

50

55

60

65

70

Control Input

Time [s]

Time [s]




38


0 10 20 30 40 50 60

0

0 10 20 30 40 50 6040

45

50

55

60

65

70

Control Input

Time [s]

Time [s]




38


0 10 20 30 40 50 60

0

0 10 20 30 40 50 6040

45

50

55

60

65

70

Control Input

Time [s]

Time [s]




38


0 10 20 30 40 50 60

0

0 10 20 30 40 50 6040

45

50

55

60

65

70

Control Input

Time [s]

Time [s]




38

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]

Figure 3.10: AUV real state, and EKF state estimate, using the LTV model and constantinput weight. Initial condition of 20 meters offset from the reference.

39

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


39

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


39

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


39


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0


Figure 3.11: Case with LTV model and constant input weight. Initial condition of 20meters offset from the reference.

40


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0



40


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0



40


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0



40

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]

Figure 3.12: AUV real state, and EKF state estimate, using the LTV model and statedependent input weight. Initial condition of 20 meters offset from the reference.

41

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


41

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


41

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


41


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0


Figure 3.13: Case with LTV model and state dependent input weight. Initial condition of20 meters offset from the reference.

42


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0



42


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0



42


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0



42

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]

Figure 3.14: AUV real state, and its EKF state estimate, using the LTV model and constantinput weight. Initial condition of 50 meters offset from the reference.

43

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


43

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


43

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


43


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0



44


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0



44


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0



44


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0



44

Simulation results

0 10 20 30 40 50 602.5

3


u,

no

t es

tim

ated

!

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


45

Simulation results

0 10 20 30 40 50 602.5

3


u,

no

t es

tim

ated

!

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


45

Simulation results

0 10 20 30 40 50 602.5

3


u,

no

t es

tim

ated

!

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


45

Simulation results

0 10 20 30 40 50 602.5

3


u,

no

t es

tim

ated

!

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


45


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140

0

10



46


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140

0

10



46


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140

0

10



46


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140

0

10



46

Simulation results

0 10 20 30 40 50 602

3

4State (red), State Estimate (blue)

u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0xf

Time [s]


47

Simulation results

0 10 20 30 40 50 602

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0xf

Time [s]


47

Simulation results

0 10 20 30 40 50 602

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0xf

Time [s]


47

Simulation results

0 10 20 30 40 50 602

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0xf

Time [s]


47


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120

0

20



48


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120

0

20



48


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120

0

20



48


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120

0

20



48

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


49

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


49

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


49

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


49


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]


0 10 20 30 40 50 60 70 80 90 100

0

20



50


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]


0 10 20 30 40 50 60 70 80 90 100

0

20



50


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]


0 10 20 30 40 50 60 70 80 90 100

0

20



50


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]


0 10 20 30 40 50 60 70 80 90 100

0

20



50

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]

Figure 3.22: AUV real state, and EKF state estimate, using the LTV-LTI model and con-stant input weight. Initial condition of 20 meters offset from the reference.

51

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


51

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


51

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


51


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0

10


Figure 3.23: Case with LTV-LTI model and constant input weight. Initial condition of 20meters offset from the reference.

52


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0

10



52


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0

10



52


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0

10



52

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]

Figure 3.24: AUV real state, and EKF state estimate, using the LTV-LTI model and statedependent input weight. Initial condition of 20 meters offset from the reference.

53

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


53

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


53

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

2

v / 2

0 10 20 30 40 50 60

0

50

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


53


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0

10


Figure 3.25: Case with LTV-LTI model and state dependent input weight. Initial conditionof 20 meters offset from the reference.

54


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0

10



54


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0

10



54


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

5

10

15

20

25

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140 160 180

0

10



54

Simulation results

0 10 20 30 40 50 602

3


u

0 10 20 30 40 50 60

0

5

0 10 20 30 40 50 60

0

5

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0xf

Time [s]

Figure 3.26: AUV real state, and its EKF state estimate, using the LTV-LTI model andconstant input weight. Initial condition of 50 meters offset from the reference.

55

Simulation results

0 10 20 30 40 50 602

3


u

0 10 20 30 40 50 60

0

5

0 10 20 30 40 50 60

0

5

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0xf

Time [s]


55

Simulation results

0 10 20 30 40 50 602

3


u

0 10 20 30 40 50 60

0

5

0 10 20 30 40 50 60

0

5

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0xf

Time [s]


55

Simulation results

0 10 20 30 40 50 602

3


u

0 10 20 30 40 50 60

0

5

0 10 20 30 40 50 60

0

5

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0xf

Time [s]


55


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]




56


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]




56


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]




56


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]




56

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


57

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


57

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


57

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

100

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


57


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140

0

10



58


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140

0

10



58


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140

0

10



58


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

20

40

60

Control Input

Time [s]

Time [s]


0 20 40 60 80 100 120 140

0

10



58

Simulation results

0 10 20 30 40 50 600

2


u

0 10 20 30 40 50 60

0

5

0 10 20 30 40 50 60

0

5

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0xf

Time [s]


59

Simulation results

0 10 20 30 40 50 600

2


u

0 10 20 30 40 50 60

0

5

0 10 20 30 40 50 60

0

5

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0xf

Time [s]


59

Simulation results

0 10 20 30 40 50 600

2


u

0 10 20 30 40 50 60

0

5

0 10 20 30 40 50 60

0

5

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0xf

Time [s]


59

Simulation results

0 10 20 30 40 50 600

2


u

0 10 20 30 40 50 60

0

5

0 10 20 30 40 50 60

0

5

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0xf

Time [s]


59


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]



Figure 3.31: Case with LTV-LTI model and constant input weight. Initial condition of100 meters offset from the reference.

60


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]




60


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]




60


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]




60

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


61

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


61

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


61

Simulation results

0 10 20 30 40 50 602.5

3


u

0 10 20 30 40 50 60

0

2

0 10 20 30 40 50 60

0

2

q

0 10 20 30 40 50 60

0

5

v / 2

0 10 20 30 40 50 60

0

200

k

0 10 20 30 40 50 60

0

0.05

xf

Time [s]


61


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]


0 10 20 30 40 50 60 70 80

0

20



62


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]


0 10 20 30 40 50 60 70 80

0

20



62


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]


0 10 20 30 40 50 60 70 80

0

20



62


0 10 20 30 40 50 60

0

0 10 20 30 40 50 60

0

50

100

150

Control Input

Time [s]

Time [s]


0 10 20 30 40 50 60 70 80

0

20



62

Simulation results

0 10 20 30 40 50 600

50

100

150

200

250Hessian Condition Number: ||H || ||H||

Time [s]

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 250

100

150

200

250Magnification

Time [s]

(a) Constant input weight.

0 10 20 30 40 50 600

20

40

60

80

100

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 21

2

3

4

5

Hessian Condition Number: ||H || ||H||

Time [s]

Magnification

Time [s]

(b) State dependent input weight.

Figure 3.34: Hessian Condition Numbers, when using the LTV model. Initial conditionof 100 meters offset from the reference.

63

Simulation results

0 10 20 30 40 50 600

50

100

150

200


Time [s]

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 250

100

150

200

250Magnification

Time [s]


0 10 20 30 40 50 600

20

40

60

80

100

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 21

2

3

4

5


Time [s]

Magnification

Time [s]



63

Simulation results

0 10 20 30 40 50 600

50

100

150

200


Time [s]

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 250

100

150

200

250Magnification

Time [s]


0 10 20 30 40 50 600

20

40

60

80

100

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 21

2

3

4

5


Time [s]

Magnification

Time [s]



63

Simulation results

0 10 20 30 40 50 600

50

100

150

200


Time [s]

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 250

100

150

200

250Magnification

Time [s]


0 10 20 30 40 50 600

20

40

60

80

100

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 21

2

3

4

5


Time [s]

Magnification

Time [s]



63


0 10 20 30 40 50 6050

60

70

80

90

100

110

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 250

60

70

80

90

100

110


Time [s]

Magnification

Time [s]


0 10 20 30 40 50 600

20

40

60

80

100

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 215

20

25

30

35


Time [s]

Magnification

Time [s]



64


0 10 20 30 40 50 6050

60

70

80

90

100

110

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 250

60

70

80

90

100

110


Time [s]

Magnification

Time [s]


0 10 20 30 40 50 600

20

40

60

80

100

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 215

20

25

30

35


Time [s]

Magnification

Time [s]



64


0 10 20 30 40 50 6050

60

70

80

90

100

110

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 250

60

70

80

90

100

110


Time [s]

Magnification

Time [s]


0 10 20 30 40 50 600

20

40

60

80

100

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 215

20

25

30

35


Time [s]

Magnification

Time [s]



64


0 10 20 30 40 50 6050

60

70

80

90

100

110

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 250

60

70

80

90

100

110


Time [s]

Magnification

Time [s]


0 10 20 30 40 50 600

20

40

60

80

100

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 215

20

25

30

35


Time [s]

Magnification

Time [s]



64

Discussion

3.4 Discussion

In order to clarify the contribution of this chapter, a brief introduction to the robustnessof MPC is given. In addition the use of the state dependent input weight is discussed.Note that no attempt is made at providing an exhaustive exposition of the area of robustMPC. Instead, the focus will be on MPC for constrained, linear systems, which covers themajority of industrial applications [Qin & Badgwell, 2003]. The analysis of robustness fornonlinear systems is generally much more difficult, the interested reader should consultRawlings & Mayne [2009] and references therein where some results in this direction aredescribed.

3.4.1 A brief introduction to robustness of MPC controllers

The robustness of a closed-loop system indicates the extent to which the behavior of thesystem is sensitive to uncertainty. Here the term uncertainty refers to anything that mayaffect system behavior that is unknown at the controller design stage, such as errors in themodel on which the controller design is based, unknown disturbances or measurementnoise.

In linear systems theory, it is common to distinguish between Robust Stability (RS)and Robust Performance (RP) [Zhou et al., 1996]. For a robustly stable system, uncer-tainty cannot cause the closed-loop system to become unstable. Similarly , for a systemwith robust performance uncertainty can only cause modest performance degradation.H∞ control theory provides powerful methods for the design and analysis of robustnessfor linear systems.

In MPC, a constrained optimization, such as (2.6), is solved for every time step andthe optimal input is applied to the plant. If the initial state xk is such that there is nosequence u that fulfills constraints (2.6c)-(2.6e), thus the optimization problem is denom-inated infeasible. In general the fulfillment of the constraints is desired not only at a giventime, but also at all subsequent times. In Chisci et al. [2001] it is shown how to ensurerecursive feasibility and also how this can be ensured for bounded disturbances by ap-propriately restricting the constraints. The approach also ensures robust stability in thecase of bounded disturbances. It is interesting to note that in case of non-constant distur-bances there is no guarantee that the state will converge to the origin, however it will stayswithin the neighborhood of the origin. In similar fashion, Pluymers et al. [2005] showshow to ensure robust feasibility and robust stability for systems with parametric model

65

Discussion

3.4 Discussion






65

Discussion

3.4 Discussion






65

Discussion

3.4 Discussion






65


uncertainty, where the model uncertainty is represented as

xk+1 = Akxk +Bkuk (3.38)

Ak =m∑i=1

λikAi

Bk =m∑i=1

λikBi

m∑i=1

1, λik ≥ 0. (3.39)

That is, the true plant must lie inside a polytope whose vertices are given by the ‘extreme’models (Ai,Bi). This approach may be adopted for both time-invariant and time-varyingmodels. Kothare et al. [1996] shows an approach to deal with this model uncertainty byusing Linear Matrix Inequalities.

The approaches in the papers referenced above are numerically tractable and poten-tially implementable. However, they address only robust feasibility and robust stabilityof uncertain, constrained linear systems. No attempt is made to optimize robust per-formance. Instead, nominal performance is optimized. Several authors have proposedoptimizing robust performance by solving specific min-max optimization problems, seeMayne et al. [2000] and references therein. However, these latter approaches are not prac-ticably tractable due to the computational burden, especially for online implementations[Rawlings & Mayne, 2009].

For an introduction to MPC for optimization of robust performance as well as ro-bust MPC for nonlinear systems the reader is referred to Rawlings & Mayne [2009] andreferences therein.

3.4.2 Industrial approaches to robust MPC

This brief introduction to robust MPC above refers mainly to ‘academic’ approaches.Industrial MPC controllers seem to deal with this problem in a different way [Qin &Badgwell, 2003].

Feasibility issues are generally addressed by controlling what constraints are violatedand by how much, instead of ensuring that constraints are never violated. This can be doneby:

• Solving one or more smaller optimizations prior to solving the ‘main’ MPC prob-lem. These first stage optimizations determine how much the main MPC constraintswill have to be relaxed such that a feasible solution can exist.

• Alternatively, the main MPC problem can be augmented with slack variables in the

66




Ak =m∑i=1

λikAi

Bk =m∑i=1

λikBi

m∑i=1

1, λik ≥ 0. (3.39)









66




Ak =m∑i=1

λikAi

Bk =m∑i=1

λikBi

m∑i=1

1, λik ≥ 0. (3.39)









66




Ak =m∑i=1

λikAi

Bk =m∑i=1

λikBi

m∑i=1

1, λik ≥ 0. (3.39)









66

Discussion

constraints and additional terms in the cost function to penalize constraint viola-tions.

These approaches to ensure feasibility are discussed in works such as Scokaert & Rawl-ings [1999]; Hovd & Braatz [2001]; Vada [2000].

From the literature on robustness of linear systems it is easy to obtain an intuitive un-derstanding of the robustness problems with ill-conditioned plants. Consider a Multiple-Input Multiple-Output plant y = Mu, with y output vector, u input vector. The modelM is matrix valued, that is only the steady state response is considered for simplicity ofpresentation. The condition number cn(M ) is defined as

cn(M ) =σ(M )

σ(M )(3.40)

where σ and σ are the largest and the smallest singular values, respectively.The generalization to dynamic models in the frequency domain is straightforward

if we consider M , and hence also the singular values, as functions of frequency. 1 Ifcn(M ) = 1, the gain from input to output is independent on the direction of the input,whereas a large cn(M ) indicates that the gain is strongly dependent on input direction.

For a reference vector r, the input u0 which removes the offset in the output is easilydetermined2

u0 = −M−1(y − r) (3.41)

The singular values of M−1 are the inverses of the singular values of M , which meansthat a large input will have to be applied in order to remove an offset in a direction wherethe input has a low gain. It is well known from linear algebra that a small change (error)in an ill-conditioned matrix can lead to a large change (error) in the inverse. Applied toour simple example this means that the controller will attempt to apply a large input inthe low gain directions, but due to model error there are some differences between thelow gain directions of the ‘true’ plant and the plant ‘model’. As a result, part of this largeinput will miss the low gain direction of the true plant, and instead be fed into a high gaindirection of the true plant. The result can be a large (unwanted) effect on the output. In afeedback system, such model errors can easily degrade performance and also jeopardizestability.

1Singular values are scaling dependent, so in order to use the singular values for analysis, the plantM needs to be appropriately scaled. Skogestad & Postlethwaite [2005] recommend scaling the inputsaccording to their range of actuation and the output according to the maximum acceptable offset in eachoutput.

2Here issues regarding invertibility of a dynamic model are neglected, see Skogestad & Postlethwaite[2005] for details.

67

Discussion




cn(M ) =σ(M )

σ(M )(3.40)




u0 = −M−1(y − r) (3.41)




67

Discussion




cn(M ) =σ(M )

σ(M )(3.40)




u0 = −M−1(y − r) (3.41)




67

Discussion




cn(M ) =σ(M )

σ(M )(3.40)




u0 = −M−1(y − r) (3.41)




67


Consider now (2.6) - (2.7). It is possible to express the predicted future states as⎡⎢⎣

xk+1...

xk+Np

⎤⎥⎦ = Axk + B

⎡⎢⎣

uk...

uk+Np−1

⎤⎥⎦ . (3.42)

Recalling the QP form (2.8), in which the Hessian has the following structure [Ma-ciejowski, 2002]

H = BT QB + R (3.43)

where

B =

⎡⎢⎢⎢⎢⎢⎢⎣

B 0 . . . . . . 0

AB +B B 0...

... . . . . . . . . . ...

... . . . B 0∑Np−1i=0 AiB . . . . . . AB +B B

⎤⎥⎥⎥⎥⎥⎥⎦, (3.44)

and

Q =

⎡⎢⎢⎢⎣Q 0 . . . 0

0. . . . . . ...

... . . . . . . 00 . . . 0 Q

⎤⎥⎥⎥⎦ , (3.45)

R =

⎡⎢⎢⎢⎣R 0 . . . 0

0. . . . . . ...

... . . . . . . 00 . . . 0 R

⎤⎥⎥⎥⎦ , (3.46)

and the Gradient is instead

G = BT QA (3.47)

where

A =

⎡⎢⎢⎢⎣

AA2

...ANp

⎤⎥⎥⎥⎦ . (3.48)

68



xk+1...

xk+Np

⎤⎥⎦ = Axk + B

⎡⎢⎣

uk...

uk+Np−1

⎤⎥⎦ . (3.42)


H = BT QB + R (3.43)

where

B =

⎡⎢⎢⎢⎢⎢⎢⎣

B 0 . . . . . . 0

AB +B B 0...

... . . . . . . . . . ...

... . . . B 0∑Np−1i=0 AiB . . . . . . AB +B B

⎤⎥⎥⎥⎥⎥⎥⎦, (3.44)

and

Q =

⎡⎢⎢⎢⎣Q 0 . . . 0

0. . . . . . ...

... . . . . . . 00 . . . 0 Q

⎤⎥⎥⎥⎦ , (3.45)

R =

⎡⎢⎢⎢⎣R 0 . . . 0

0. . . . . . ...

... . . . . . . 00 . . . 0 R

⎤⎥⎥⎥⎦ , (3.46)


G = BT QA (3.47)

where

A =

⎡⎢⎢⎢⎣

AA2

...ANp

⎤⎥⎥⎥⎦ . (3.48)

68



xk+1...

xk+Np

⎤⎥⎦ = Axk + B

⎡⎢⎣

uk...

uk+Np−1

⎤⎥⎦ . (3.42)


H = BT QB + R (3.43)

where

B =

⎡⎢⎢⎢⎢⎢⎢⎣

B 0 . . . . . . 0

AB +B B 0...

... . . . . . . . . . ...

... . . . B 0∑Np−1i=0 AiB . . . . . . AB +B B

⎤⎥⎥⎥⎥⎥⎥⎦, (3.44)

and

Q =

⎡⎢⎢⎢⎣Q 0 . . . 0

0. . . . . . ...

... . . . . . . 00 . . . 0 Q

⎤⎥⎥⎥⎦ , (3.45)

R =

⎡⎢⎢⎢⎣R 0 . . . 0

0. . . . . . ...

... . . . . . . 00 . . . 0 R

⎤⎥⎥⎥⎦ , (3.46)


G = BT QA (3.47)

where

A =

⎡⎢⎢⎢⎣

AA2

...ANp

⎤⎥⎥⎥⎦ . (3.48)

68



xk+1...

xk+Np

⎤⎥⎦ = Axk + B

⎡⎢⎣

uk...

uk+Np−1

⎤⎥⎦ . (3.42)


H = BT QB + R (3.43)

where

B =

⎡⎢⎢⎢⎢⎢⎢⎣

B 0 . . . . . . 0

AB +B B 0...

... . . . . . . . . . ...

... . . . B 0∑Np−1i=0 AiB . . . . . . AB +B B

⎤⎥⎥⎥⎥⎥⎥⎦, (3.44)

and

Q =

⎡⎢⎢⎢⎣Q 0 . . . 0

0. . . . . . ...

... . . . . . . 00 . . . 0 Q

⎤⎥⎥⎥⎦ , (3.45)

R =

⎡⎢⎢⎢⎣R 0 . . . 0

0. . . . . . ...

... . . . . . . 00 . . . 0 R

⎤⎥⎥⎥⎦ , (3.46)


G = BT QA (3.47)

where

A =

⎡⎢⎢⎢⎣

AA2

...ANp

⎤⎥⎥⎥⎦ . (3.48)

68

Discussion

In the case of no active constraints, the optimal input sequence for MPC can easilybe found by differentiating the criterion in (2.8a), and solving for the input sequence. Thisgives

uk =

⎡⎢⎣

uk...

uk+Np−1

⎤⎥⎦ = −H−1Gxk =

(BT QB + R

)−1

BT QAxk (3.49)

It is clear that if the Hessian matrix H is ill-conditioned, a small error in H (stemmingfrom a small model error) can lead to a large error in H−1, and hence the effect of theinput sequence may be far from excepted, with strong detrimental effect on control perfor-mance. Industrial MPC controllers often address this problem in one of the two followingways:

• Using Singular Value Thresholding (SVT), where the plant model (that is, B) is de-composed using Singular Value Decomposition (SVD), and small singular valuesare set to zero, and an approximate model B is assembled after setting the smallsingular values to zero. The approximate model is then used when solving the MPCoptimization problem. Inspecting (3.49), it is clear that singular value thresholdingdoes not actually reduce the ill-conditioning of the Hessian. Instead, the predic-tions are effectively modified such that the controller ‘does not see’ future offsetsin directions where the inputs have little effect. SVT is used in Honeywell’s RobustMultivariable Predictive Control Technology (RMPCT) [Qin & Badgwell, 2003].

• Increasing the input weight R (that is, the weight on all inputs) reduces the ill-conditioning of the Hessian, and thus avoids excessive input moves. This techniqueis used in Aspentech’s DMCplus controller [Qin & Badgwell, 2003].

It should be noted that SVT deliberately introduces a model error, whereas increasingthe input weight detunes the controller and makes nominal response slower. Thus, bothapproaches reduce nominal performance. The use of these techniques in industrial con-trollers therefore reflects the fact that most MPC controllers in industry are implementedon open-loop stable systems. These two ’robustification techniques’ are studied and com-pared in Aoyama et al. [1997].

3.4.3 The state dependent input weightConsidering the brief introduction to the robustness of MPC controllers presented above,it is clear that the use of the state dependent input weight is a refinement of the indus-trially popular technique of increasing the input weight. Making the input weight statedependent (when properly applied) will, however, only reduce the controller gain whenthis is required by the operating conditions, and will not reduce performance in ’safe’ op-erating regions. From this point of view, the state dependent input weight introduces gainscheduling into the MPC controller, as claimed in the introduction to this chapter.

69

Discussion


uk =

⎡⎢⎣

uk...

uk+Np−1

⎤⎥⎦ = −H−1Gxk =

(BT QB + R

)−1

BT QAxk (3.49)






69

Discussion


uk =

⎡⎢⎣

uk...

uk+Np−1

⎤⎥⎦ = −H−1Gxk =

(BT QB + R

)−1

BT QAxk (3.49)






69

Discussion


uk =

⎡⎢⎣

uk...

uk+Np−1

⎤⎥⎦ = −H−1Gxk =

(BT QB + R

)−1

BT QAxk (3.49)






69


The state dependent input weight requires that the control engineer has a sound un-derstanding of the control problem, and implementing it in standard MPC controllers(with a quadratic objective function) is very simple. Also, the weight calculation resultsin a negligible increase in computational cost1.

3.5 ConclusionsThe industrial ‘robustification techniques’, described in Section 3.4.2, are motivated asways of avoiding drastic performance degradation stemming from small model errors.For the AUV example, it seems clear that these model errors stem from using a linearizedmodel of the AUV. The linearized model used in the MPC produces then an ill-conditionedoptimization problem. This generates performance degradation, and in some cases thecontrol fails to reach its goal.

A model predictive control framework for the depth control of an underwater vehicleis presented where different MPC approaches are implemented. It is shown how by usingan LTV model, and the state dependent input weight it is possible to improve the regionof attraction of the controller and have a smooth control action. The nonlinear inputweight definition is inspired by the gain-scheduling technique. It helps to better adapt thecontrol response to the initial condition (depth of the AUV). In fact, although the system isnonlinear, the state dependent input weight allows the use of linear models for prediction,giving the best performance with the LTV model. Related to this gain-scheduling effect isthe reduction in the condition number of the Hessian that results from the state dependentinput weight. As discussed, reducing the ill-conditioning of the Hessian is a commonapproach to improve the robustness of industrial MPC. This is of particular importancefor this problem when the AUV is far from the reference and large inputs would otherwisebe demanded.

An alternative approach is to solve the nonlinear model equations, which wouldtypically require solving a Sequential Quadratic Programming (SQP) problem 2 (Nocedal& Wright [2000]), instead of solving a single QP problem at each time step, as brieflyintroduced in Section (2.3.3). Such an approach may be computationally tractable, but thecomputational cost (and difficulty of coding) is much higher than for the state dependentinput weight, which requires solving only a single QP at each time step. In addition,even if all available physical knowledge may be applied when formulating the nonlinearmodel, all models are in practice uncertain to some extent. Thus, although direct use of thenonlinear model removes inaccuracies stemming from model linearization, the resulting

1Actually, for ordinary MPC problems where the optimization problem is solved online, the state depen-dent input weight probably saves online computing compared to solving QP problems with ill-conditionedHessians.

2That is, as sequence of QP problems that gradually converge to the solution of the nonlinear problem.

70








70








70








70

Conclusions

MPC controller may nevertheless still experience robustness problems.In summary, the state dependent input weight is clearly related to approaches in

use in industry to enhance robustness of MPC controllers. Although more sophisticatedmethods exist in the academic literature, there seems to be little need for such methodsfor the problem studied here. The main advantages of using the state dependent inputweight is the low computational cost and easy implementation, compared with min-maxrobustification techniques, or SQP, for example. With the state dependent approach thecontroller gain is increased smoothly and automatically when entering the safe operatingregion. The main disadvantage is the need for the control engineer to know where the safeoperating region is, in order to be able to reliably design the state dependent input weight.

71

Conclusions



71

Conclusions



71

Conclusions



71


72


72


72


72

Chapter 4

Persistently Exciting Model PredictiveControl, a dual control approach

In this chapter, MPC is considered in an adaptive context where the model adjustmentrequirements are built into the MPC formulation. This is a general question of dealingwith the dual adaptive control problem, discussed in Section 4.1.1. The input has the dualpurpose of regulating the system via feedback and providing the excitation necessary forthe identification and adaptation of the system. It is well known that these two purposesare generally in conflict. Further, it is well known that attempting to optimize a controlperformance criterion over both the regulation and excitation frameworks is essentially anintractable problem. However, several interesting results have been obtained by approxi-mating the original dual control problem.

Fundamentally, MPC is a non-dynamic or memoryless state feedback control. Be-cause of its use of a model, MPC should be amenable to adaptive implementation andto online tuning of the model. Such an approach requires guaranteeing signal properties,known as ‘persistent excitation’, to ensure uniform identifiability of the model, often ex-pressed in terms of spectral content or ‘sufficient richness’ of a periodic input. Thus, anapproach to augment the input constraint set of MPC to provide this guarantee is pro-posed. This requires equipping the controller with its own state to capture the controlsignal history. The feasibility of periodic signals for this condition is also established, andseveral computational examples are presented to illustrate the technique and its properties.

The approach taken here is to augment the constrained optimization of MPC to in-clude an additional constraint to ensure that a minimal level of excitation is preserved. Thenormal statement of the persistence of excitation condition (PEC) is via uniform positivedefinite matrix bounds on the running summed outer products of the regressor sequencein the system identification. In the context of open-loop, stationary, linear system iden-tification of models with N parameters to be identified, this excitation requirement isreinterpreted as the input signal sufficient richness condition (SRC) that the vectors com-

73

Chapter 4





73

Chapter 4





73

Chapter 4





73

4. Persistently Exciting Model Predictive Control, a dual control approach

posed of N delayed values of the input signal themselves satisfy the PEC. In turn andstill for stationary scalar systems, this condition can be converted to a requirement thatthe input signal contains at least N distinct complex frequency components [Goodwin &Sin, 1984], that is, as many complex sinusoids as there are parameters to identify. Al-though the terms PEC and SRC are used almost interchangeably [Green & Moore, 1986],SRC is reserved here for the PEC property pertaining to vectors composed solely of themanipulable input signal.

For closed-loop system identification the regressor PEC requirement is normallytransferred to the reference input signal and requires the stability and/or stationarity ofthe closed-loop to achieve a signal upper bound. An equivalent condition to ensure theconvergence of system identification using pseudo-linear regression is that the input signalto the plant yields a persistently exciting pseudo-regressor sequence, in order for the pa-rameter estimation error algorithm to converge sufficiently rapidly. For open-loop outputerror schemes applied to minimal systems satisfying the standard positive real condition,Anderson & Johnson Jr [1982] show that input richness conditions alone suffice for theentire regressor to satisfy the excitation condition. Accordingly, for many standard linearsystem identification schemes, the uniform convergence of the parameter estimator maybe ensured provided the plant input signal {uk} satisfies a sufficient richness property.The development in this chapter, denominated Persistently Exciting MPC (PE-MPC), isto include an input sufficient richness constraint directly into the MPC formulation. In theabsence of prevailing signal conditions in the MPC-controlled loop yielding excitation,this constraint becomes active to ensure plant identifiability in closed-loop. It is shownthat, in this circumstance, periodic inputs are feasible.

Due to the focus of MPC on stabilization and regulation, the controller does not needto inherently produce a plant input signal {uk} that satisfies the SRC. Indeed, the in-clusion of input richness into PE-MPC, since it adds another constraint, could diminishregulation performance in line with expectations from dual adaptive control. Differentapproaches have been proposed earlier to ensure richness. For example, in Sotomayoret al. [2009] it is shown that by incorporating an additive external signal, also known asa dithering signal, it is possible to obtain sufficiently rich closed-loop signals for pro-cess identifiability. However, in an MPC framework, an unfortunate choice of ditheringsignal might cause constraint violation. For example, after the manipulated variable iscalculated, if the magnitude of a dithering signal is large enough, some input or outputconstraint may be violated. One might accommodate the dither by tightening the con-straints to conservative values. However, the dither must diminish control performanceeven when the closed-loop signals are otherwise persistently exciting by dint of, say, ref-erence changes or other transients. In Genceli & Nikolaou [1996] an approach, namedMPCI, is presented. In this case, the input is forced to be persistently exciting by impos-ing a Persistently Exciting (PE) constraint. However, the MPCI has the drawback that itsformulation contains the definition of an equality constraint to enforce periodic solutions.

74





74





74





74

Background results

Another approach is Hovd & Bitmead [2004], where an augmented cost function is intro-duced in an NMPC contest to add the dual effect necessary such that state estimation isimproved.

We consider incorporating the input PEC into the formulation of an MPC controllerthrough the inclusion of an additional constraint, guaranteeing a sufficiently rich inputsignal but with the possibility of this constraint being inactive during normal systemtransients. In the stationary case, this approach admits, rather than imposes, a periodicinput. Since MPC involves the receding horizon computation at time k of a plant-state-dependent control sequence, {uk,uk+1, . . . ,uk+Np−1}, of which only the initial element,uk, is applied. The remaining elements are relegated to the initial feasible solution ofthe subsequent solution and sufficient richness asserted on the forward-looking Np-stepMPC solution need not occur as result, because of the optimization problem restatementafter the application of the first element. Thus, it is necessary to introduce a state intothe MPC controller to effectively look backwards to ensure that uk satisfies the SRC withrespect to its immediate predecessor inputs. This is a novelty when compared to the for-mulation of MPC as a non-dynamic (memoryless) function of the plant state; PE-MPC isnow a dynamic plant state feedback control law. By the same token, our formulation ofPE-MPC associates this new constraint only with the first manipulated variable uk. Thishas a direct consequence on the analysis of the constraint, simplifying the solution of thecorresponding optimization problem. Finally, we also need to assume for our analysis, aswith most early approaches to adaptive control, that other mechanisms are at play to en-sure the closed-loop system is stabilized so that an upper bound is available for the inputto ensure that the PEC is a feasible constraint for the MPC problem. That is, our methodswill need to assume and rely on the effectiveness and feasibility of the regulation side ofthe MPC controller.

In this chapter, we shall denote by: n the plant system state dimension, m the inputdimension, p the output dimension, Np the control and prediction horizon of the MPC, Nthe number of parameters to be identified in a plant model, P the input sufficient richnessbackwards-looking horizon to be defined.

4.1 Background results

In this section, a short general introduction to dual control is given. Then, technicalconcepts are defined and introduced to arrive at the main results of the chapter. Thesebackground results are fundamental to understand how the Persistently Exciting ModelPredictive Control formulation is obtained.

75

Background results






75

Background results






75

Background results






75


4.1.1 Fundamentals of Dual Control theoryIn general, control systems are designed to spend as little control effort as possible in orderto obtain the required control performance. That is, a controller that achieves its perfor-mance specifications while using the minimum amount of energy is wanted. However,when we have to learn from a system, it is necessary to have a certain level of excitation,such that the ‘learning procedure’ can work properly. To obtain a minimum level of exci-tation, a certain level of energy has to be used. Thus, it is obvious that control action andlearning action are in conflict. The compromise between these two actions leads to thedual control problem.

The dual control problem was first defined and introduced by Feldbaum at the begin-ning of the Sixties with a series of four seminal papers, [Feldbaum, 1960a,b, 1961a,b].In his work he discussed the problem of conflict between control quality and informationgathering. Feldbaum introduced the dual effect and neutral system notions, discussedmore in detail in next section. He defined the dual control system such as a system char-acterized by having a dual effect.

It is difficult to have a formal definition of dual control, however as mentioned inBrustad [1991], the following definition may be given:

Definition 4.1.1. Dual controller:A dual controller is a model-based controller where the model is, to some degree, uncer-tain, and where the controller in addition to its usual control task, spends some energy inreducing the model uncertainties (learning) in such a way that the overall control perfor-mance is improved.

This is a general definition which covers a large range of controllers. The scope ofthis chapter is to describe the dual control problem in a general manner. It is hoped thatthis introduction to dual control will make easier to appreciate the results in the chapter.

In 2000, IEEE Control System Society listed dual control as one of the 25 mostprominent subjects in the last century which had a great impact on control theory devel-opment [Li et al., 2005].

In principle, the optimal solution of a dual control problem can be found by dynamicprogramming, by solving the Bellman functional equation. However, it is well knownthat is very difficult to find an analytical recursive solution, and the problem is generallyconsidered intractable [Kumar & Varaiya, 1986]. Moreover, the dimensionality of theunderling spaces causes several numerical problems that makes this problem practicallyunsolvable [Bayard & Eslami, 1985], [Bar-Shalom & Tse, 1976].

76









76









76









76

Background results

Dual effect and neutral systems

The principal characteristic of a dual controller is to have a dual effect. Feldbaum de-scribed the dual effect as the application of the following two actions,

• directing, and• investigating.

In Bar-Shalom [1981] both features are described as caution and probing, respec-tively. In more detail, a dual controller is capable of

• driving the system towards a desired state (directing);• performing some learning procedure such that system uncertainties are reduced (in-

vestigating).

Typically, these two aspects are in conflict, and they are source of difficulties when findingthe optimal dual control solution.

It is worth mentioning that not every system can have a dual effect. In fact it is pos-sible to define a neutral system [Bar-Shalom & Tse, 1974].

Definition 4.1.2. Neutral system:A system that does not have a dual effect is called neutral.

A practical way to understand whether a system is neutral or not is to check that theuncertainty in the system is totally independent from the past control sequence.

In Brustad [1991] an example of neutral system is given. It is shown that a linearGaussian system does not have dual effect. Consider the problem of estimating the stateof the following linear system

xk+1 = Axk + vk

yk = Cxk +wk (4.1)

where vk and wk are white Gaussian noise distributed signals, with covariances P v andP w, respectively. The uncertainty in the system is expressed by the state covariancematrix P x. Using the standard Kalman Filter (see Chapter 5) the covariance is computedby

Kk = P xk|k−1C

T (CP xk|k−1C + P w)−1 (4.2)

P xk|k = (I −KkC)P x

k|k−1 (4.3)

P xk+1|k = AP x

k|kAT + P v (4.4)

77

Background results






vestigating).






xk+1 = Axk + vk

yk = Cxk +wk (4.1)


Kk = P xk|k−1C

T (CP xk|k−1C + P w)−1 (4.2)


k|k−1 (4.3)

P xk+1|k = AP x

k|kAT + P v (4.4)

77

Background results






vestigating).






xk+1 = Axk + vk

yk = Cxk +wk (4.1)


Kk = P xk|k−1C

T (CP xk|k−1C + P w)−1 (4.2)


k|k−1 (4.3)

P xk+1|k = AP x

k|kAT + P v (4.4)

77

Background results






vestigating).






xk+1 = Axk + vk

yk = Cxk +wk (4.1)


Kk = P xk|k−1C

T (CP xk|k−1C + P w)−1 (4.2)


k|k−1 (4.3)

P xk+1|k = AP x

k|kAT + P v (4.4)

77


Thus, as it can be seen, A, C, P v, and P w are all independent of the input uk. Therefore,the state covariance matrix P x is totally independent from the past input sequence, andthus the system (4.1) is neutral.

Formulation

In Filatov & Unbehauen [2004] a general formulation of the dual control problem ispresented. The formal solution using Stochastic Dynamic Programming is given, andthe main difficulties on finding the optimal solution are discussed. It is shown how toimprove (but not optimize) the performance, using the dual effect, of different adaptivecontrol problems without excessively complicating the original algorithms, such that it ispossible to have practical online implementations.

According to Filatov & Unbehauen [2004], let us define the following dual controlproblem. Consider the state space representation

xk+1 = f(xk,uk,ωk;θk) (4.5a)yk = h(xk,vk;θk) (4.5b)

where xk ∈ Rn, uk ∈ Rm, yk ∈ Rp, ωk and vk are sequences of independent zero-meanrandom variables, and θk ∈ RN denotes the vector of unknown parameters, with assumedtime-varying dynamic

θk+1 = p(θk, εk) (4.6)

where the function p(·) describes the time-varying nature of the parameter vector θ, andεk is a vector of independent random white noise sequence with zero-mean and knownprobability distribution.

Control inputs and outputs, available at time k, are denoted by

Υk = {uk−1,uk−2, . . . ,u0,yk,yk−1, . . . ,y0} (4.7)

for k = 1, . . . , Np − 1 and Υ0 = {y0}.The performance index

J = E

{Np−1∑k=0

g(xk+1,uk)

}(4.8)

is used for control optimization purpose, where g(·) is a positive, convex, scalar function.The operator E{·} denotes the expectation, and it is taken with respect to all the randomvariables x,θ,ω,v, ε.

The dual control problem consists of finding the control policy

uk(Υk) ∈ Ωk, k = 0, 1, . . . , Np − 1 (4.9)

78



Formulation





θk+1 = p(θk, εk) (4.6)



Υk = {uk−1,uk−2, . . . ,u0,yk,yk−1, . . . ,y0} (4.7)


J = E

{Np−1∑k=0

g(xk+1,uk)

}(4.8)



uk(Υk) ∈ Ωk, k = 0, 1, . . . , Np − 1 (4.9)

78



Formulation





θk+1 = p(θk, εk) (4.6)



Υk = {uk−1,uk−2, . . . ,u0,yk,yk−1, . . . ,y0} (4.7)


J = E

{Np−1∑k=0

g(xk+1,uk)

}(4.8)



uk(Υk) ∈ Ωk, k = 0, 1, . . . , Np − 1 (4.9)

78



Formulation





θk+1 = p(θk, εk) (4.6)



Υk = {uk−1,uk−2, . . . ,u0,yk,yk−1, . . . ,y0} (4.7)


J = E

{Np−1∑k=0

g(xk+1,uk)

}(4.8)



uk(Υk) ∈ Ωk, k = 0, 1, . . . , Np − 1 (4.9)

78

Background results

that minimizes the performance index (4.8) for the system (4.5), where the unknown pa-rameter dynamics is described in (4.6). The optimal control sequence has to be containedin Ωk, that defines the admissible control values domain, in the space Rm.

To find the optimal solution (4.9), stochastic dynamic programming is used [Kall &Wallace, 1994; Birge & Louveax, 1997]. Thus the backward recursion

JCLONp−1(ΥNp−1) = min

uNp−1∈ΩNp−1

[E{g[xNp ,uNp−1]|ΥNp−1

}], (4.10)

JCLOk (Υk) = min

uk∈Ωk

[E{g[xk+1,uk] + JCLO

k+1 (Υk+1)|Υk

}], (4.11)

for k = Np − 2, Np − 3, . . . , 0 is obtained. The superscript CLO means Closed-LoopOptimal, that is the current control signal is obtained by considering knowledge aboutdynamic, probability statistics, and input/output measurements, in accordance to Bar-Shalom & Tse [1976] formulation. A simple recursive solution of (4.10) and (4.11) isdifficult to find due to both analytical difficulties and numerical issues of the problemdimensionality. Therefore, even for small size, simple problems it is, with few excep-tions, practically impossible to find solutions [Bar-Shalom & Tse, 1976; Bayard & Es-lami, 1985]. One notable exception is Sternby [1976], who presents a simple example ofa dual control problem with an analytical solution. However, the dual control approach istrying to simplify the problem such that a possible numerical solution is found. Thus, theanalysis of this formulation may lead to insights that can help identify the main dual prop-erties. This can then be used to define a simpler problem, which is an approximation, butcontains the dual effect. The advantage is that the new problem is practically solvable, asin Wittermark [2002], for example. Note that, however, the solution of an approximateddual control problem is typically sub-optimal rather than truly optimal.

An example of an application of dual effect, probing and controlling, is in Allisonet al. [1995], where it is shown how to implement a dual control strategy for a wood chiprefiner. A more recent application to a coupled multivariable process for paper coating isshown in Ismail et al. [2003], where the authors implement an adaptive dual control law,and a Kalman Filter for estimating the variable process gain. Moreover, the controlleris run for a trial period in the mill with successful results. This is an example of howadding the probing effect, in addition to the control effect, can substantially improve theproduction.

Difficulties in finding the optimal solution of the dual control problem (4.10-4.10)have contributed to discovering several approaches for deriving simpler sub-optimal dualcontrollers. As mentioned in Wittermark [1995], some approximated dual controllers maybe obtained by constraining the variance of parameter estimates, using serial expansion, oreven modification, of the performance index (4.8), using ideas from robust control design.Other approximation policies are discussed in Filatov & Unbehauen [2004], where anexhaustive classification of sub-optimal dual controllers is given.

79

Background results




uNp−1∈ΩNp−1


}], (4.10)

JCLOk (Υk) = min

uk∈Ωk


k+1 (Υk+1)|Υk

}], (4.11)




79

Background results




uNp−1∈ΩNp−1


}], (4.10)

JCLOk (Υk) = min

uk∈Ωk


k+1 (Υk+1)|Υk

}], (4.11)




79

Background results




uNp−1∈ΩNp−1


}], (4.10)

JCLOk (Υk) = min

uk∈Ωk


k+1 (Υk+1)|Υk

}], (4.11)




79


Fundamentally, dual control approximations may be gathered into two distinct cate-gories.

• Implicit Dual Control• Explicit Dual Control

The first category contains all methods obtained by approximating the original dualcontrol problem (4.10-4.10), or the performance index (4.8). The main characteristic ofthese controllers is that they provide good performance, but due to a significant computa-tional requirement their application in real time is practically restricted.

The second category contains all ad-hoc controllers, and all methods based on asort of bi-criterial approach. The bi-criterial approach can be obtained by modifying theperformance index (4.8), such as two distinct parts may be defined

J = J c + Je (4.12)

where J c is the cost function term that contributes to the ‘control’ part of the resultingcontroller, while Je is the cost function term that contributes to the ‘excitation’ levelof the resulting control signal. This particular form of the performance index (or costfunction) makes it possible to explicitly enforce the dual effect.

The persistently exciting model predictive control formulation given in this chaptercan be classified as an explicit dual controller.

4.1.2 Schur complementGiven a matrix

M =

[P QR S

], (4.13)

with nonsingular trailing principal square sub-matrix S, the Schur complement is definedas

M/S = P −QS−1R. (4.14)

The notation M/S indicates the Schur complement of S in M .

Definition 4.1.3. Inertia of Hermitian MatricesThe inertia of an n× n Hermitian matrix H is the ordered triple

In(H) := (p(H), q(H), z(H)) (4.15)

where p(H), q(H), and z(H) are the numbers of positive, negative, and zero eigenvaluesof H including multiplicity, respectively. The rank is rank(H) = p(H) + q(H).

80






J = J c + Je (4.12)




M =

[P QR S

], (4.13)


M/S = P −QS−1R. (4.14)



In(H) := (p(H), q(H), z(H)) (4.15)


80






J = J c + Je (4.12)




M =

[P QR S

], (4.13)


M/S = P −QS−1R. (4.14)



In(H) := (p(H), q(H), z(H)) (4.15)


80






J = J c + Je (4.12)




M =

[P QR S

], (4.13)


M/S = P −QS−1R. (4.14)



In(H) := (p(H), q(H), z(H)) (4.15)


80

Background results

Theorem 4.1.1 (Zhang [2005]). Let H be a Hermitian matrix and let S be a nonsingularprincipal sub-matrix of H . Then

In(H) = In(S) + In(H/S). (4.16)

A direct consequence of this theorem is the following.

Theorem 4.1.2. Let H be a Hermitian matrix partitioned as

H =

[H11 H12

H∗12 H22

](4.17)

where H22 is square and nonsingular. Then H > 0 if and only if both H22 > 0 andH/H22 > 0.

4.1.3 Persistence of excitation for adaptive control schemesIn general, for uniform convergence of system identification schemes it is desired that theregressor vectors generated in the time-varying closed-loop must be persistently exciting,i.e. summed outer products of the regression vector, ψk, should satisfy uniform bounded-ness conditions.

Definition 4.1.4. [Goodwin & Sin, Sec 3.4 Goodwin & Sin [1984]] A sequence of vectors{ψk : k = 1, 2, . . . } in RN is persistently exciting if there exist real numbers α and β andinteger P such that

0 < αIN ≤k0+P∑

k=k0+1

ψTk ψk ≤ βIN < ∞, ∀k0.

This is not easily verifiable, instead it is more convenient to investigate how thepersistence of excitation propagates from input signal to the regressors. This problem hasbeen studied by a number of researchers and follows if the regressor vectors are reachablefrom the input. For Single-Input, Single-Output (SISO) cases, the authors in Anderson &Johnson Jr [1982]; Mareels [1985]; Dasgupta et al. [1990] have derived conditions on theinput that guarantee parameter convergence. For multivariable cases, results have beenobtained in Green & Moore [1986]; Moore [1982]; Bai & Sastry [1985].

In Goodwin & Sin [1984] general definitions on persistence of excitation of a signalare given. We choose the following definition from Bai & Sastry [1985] and adopt theconvention from Green & Moore [1986] of referring to regressors as satisfying PEC andinputs satisfying the SRC.

81

Background results


In(H) = In(S) + In(H/S). (4.16)



H =

[H11 H12

H∗12 H22

](4.17)




0 < αIN ≤k0+P∑

k=k0+1




81

Background results


In(H) = In(S) + In(H/S). (4.16)



H =

[H11 H12

H∗12 H22

](4.17)




0 < αIN ≤k0+P∑

k=k0+1




81

Background results


In(H) = In(S) + In(H/S). (4.16)



H =

[H11 H12

H∗12 H22

](4.17)




0 < αIN ≤k0+P∑

k=k0+1




81


Definition 4.1.5. A sequence of input signals, {uk},∈ Rm is sufficiently rich of order N ,in P steps, if there exist positive integers P , ρ1 and ρ0 such that

0 < ρ0INm ≤k0+P∑

k=k0+1

⎡⎢⎢⎢⎣

uk−1

uk−2...

uk−N

⎤⎥⎥⎥⎦[uT

k−1 uTk−2 . . . uT

k−N

] ≤ ρ1INm < ∞, ∀k0.

(4.18)

This definition states that the signal uk is uniformly strongly persistently excitingin the parlance of Goodwin & Sin [1984]. Moreover, this has an interpretation in thefrequency domain. Lemma 3.4.6 of Goodwin & Sin [1984] states that a scalar (m = 1)quasi-stationary uk satisfying this definition must have a two-sided spectrum which isnon-zero at at least N frequencies.

We now state a definition and an important theorem of Green & Moore [1986], whichpermits the derivation of conditions on the input sequence {uk} to ensure persistency ofexcitation of the regressors. This problem is also very well characterized by Anderson &Johnson Jr [1982].

Consider the minimal state space system

xk+1 = Axk +Buk (4.19a)yk = Cxk +Duk (4.19b)

with xk ∈ Rn, uk ∈ Rm, yk ∈ Rp, and the associated proper transfer function

T = C(zI −A)−1B +D.

Definition 4.1.6. The system (4.19) is called output reachable if, for any y ∈ Rp andarbitrary initial state, there exists an input sequence {ui, i = 0, . . . , k < ∞} such thatits output at time k satisfies yk = y.

Definition 4.1.6 is satisfied if the following Markov parameter matrix

Mn =[D CB CAB . . . CAn−1B

],

associated with the system (4.19) has full row rank p.

Theorem 4.1.3. [Green & Moore, Green & Moore [1986]] A necessary and sufficientcondition for the output of every output reachable system (4.19) of McMillan degree nto be persistently exciting over the interval [k + 1 − n, k + l] independent of initialconditions, is that the vector ui = [uT

i ,uTi−1, . . . ,u

Ti−n]

T is sufficiently rich over theinterval [k + 1, k + l].

82



0 < ρ0INm ≤k0+P∑

k=k0+1

⎡⎢⎢⎢⎣

uk−1

uk−2...

uk−N

⎤⎥⎥⎥⎦[uT

k−1 uTk−2 . . . uT

k−N

] ≤ ρ1INm < ∞, ∀k0.

(4.18)






T = C(zI −A)−1B +D.




],



i ,uTi−1, . . . ,u

Ti−n]


82



0 < ρ0INm ≤k0+P∑

k=k0+1

⎡⎢⎢⎢⎣

uk−1

uk−2...

uk−N

⎤⎥⎥⎥⎦[uT

k−1 uTk−2 . . . uT

k−N

] ≤ ρ1INm < ∞, ∀k0.

(4.18)






T = C(zI −A)−1B +D.




],



i ,uTi−1, . . . ,u

Ti−n]


82



0 < ρ0INm ≤k0+P∑

k=k0+1

⎡⎢⎢⎢⎣

uk−1

uk−2...

uk−N

⎤⎥⎥⎥⎦[uT

k−1 uTk−2 . . . uT

k−N

] ≤ ρ1INm < ∞, ∀k0.

(4.18)






T = C(zI −A)−1B +D.




],



i ,uTi−1, . . . ,u

Ti−n]


82

Background results

Theorem 4.1.3 simply states that a sufficiently rich (of order n in l steps) input se-quence ui provides a persistently exciting output sequence. Also, any system that is notoutput reachable cannot have a persistently exciting output sequence. When adaptive es-timation is considered, the requirement is that the regressor vectors involving past inputs,past outputs and possible noise estimates are persistently exciting. The methods of Green& Moore [1986] may now be applied by defining the output reachable system to be thatwhich has input uk and output ψk, the complete regressor. Then the output being suffi-ciently rich coincides with the regressors being persistently exciting if the system is outputreachable. Thus, similar conditions to Theorem 4.1.3 are needed. For example, considerthe multivariable ARMA model

yk +A1yk−1 +A2yk−2 + · · ·+A�yk−� = B1uk−1 +B2uk−2 + · · ·+Bquk−q

with transfer matrix

T (z) = A−1(z)B(z)

where

A(z) = Iz� +A1z�−1 · · ·+A�,

B(z) = B1z�−1 + · · ·+Bqz

�−q.

Then, the associated regressor

φ�q(k) =[yTk−1 yT

k−2 . . . yTk−� uT

k−1 . . . uTk−q

]T(4.20)

must be reachable from the input, and the input itself must be persistently exciting. Tocheck the reachability of (4.20), the following result may be used.

Corollary 4.1.4. [Green & Moore Green & Moore [1986]] The regressor (4.20) is reach-able from uk if and only if [A(z) B(z)] is irreducible (has full row rank for all z) or,equivalently, A(z) and B(z) are left coprime.

When, ARMAX models are considered, pseudo-linear regressor vectors are used,instead. In this case the persistence of excitation is analyzed by Moore [1982]. Briefly,consider here

yk +A1yk−1 + · · ·+A�yk−� = B1uk−1 + · · ·+Bquk−q + ωk +C1ωk−1 + · · ·+Crωk−r

where the associated regressor is

φ�qr(k) =[yTk−1 . . . yT

k−� uTk−1 . . . uT

k−q ωTk−1 . . . ωT

k−r

]T(4.21)

83

Background results




T (z) = A−1(z)B(z)

where

A(z) = Iz� +A1z�−1 · · ·+A�,

B(z) = B1z�−1 + · · ·+Bqz

�−q.



k−2 . . . yTk−� uT

k−1 . . . uTk−q

]T(4.20)






φ�qr(k) =[yTk−1 . . . yT

k−� uTk−1 . . . uT

k−q ωTk−1 . . . ωT

k−r

]T(4.21)

83

Background results




T (z) = A−1(z)B(z)

where

A(z) = Iz� +A1z�−1 · · ·+A�,

B(z) = B1z�−1 + · · ·+Bqz

�−q.



k−2 . . . yTk−� uT

k−1 . . . uTk−q

]T(4.20)






φ�qr(k) =[yTk−1 . . . yT

k−� uTk−1 . . . uT

k−q ωTk−1 . . . ωT

k−r

]T(4.21)

83

Background results




T (z) = A−1(z)B(z)

where

A(z) = Iz� +A1z�−1 · · ·+A�,

B(z) = B1z�−1 + · · ·+Bqz

�−q.



k−2 . . . yTk−� uT

k−1 . . . uTk−q

]T(4.20)






φ�qr(k) =[yTk−1 . . . yT

k−� uTk−1 . . . uT

k−q ωTk−1 . . . ωT

k−r

]T(4.21)

83


with ωk noise estimates, generated in terms of parameter estimates

θ = {Ai, i = 1, . . . , �; Bi, i = 1, . . . , q; Ci, i = 1, . . . , r}

via

ωk = yk − yk,

= yk −[−A1yk−1 − · · · − A�yk−� + B1uk−1 + · · ·+ Bquk−q + C1ωk−1 + . . .

+ Crωk−r

].

Similarly to Corollary 4.1.4, we have the following result for ARMAX models.

Corollary 4.1.5 (Green & Moore [1986]). The pseudo-regressor (4.21) is reachable fromuk, if and only if both [A(z), B(z)], and [C(z), A(z)B(z)−A(z)B(z)] are left coprime.

4.2 Persistently Exciting Model Predictive Control

4.2.1 Derivation of constraints for persistence of excitation

To include the PEC into MPC, we observe that the MPC controller is a receding hori-zon method. That is, at the current time index k, the approach proceeds by calcu-lating the finite-horizon, open-loop, optimal constrained future control input sequence{uk

k,ukk+1, . . . ,u

kk+Np−1} for the horizon value Np.

Since this depends on the current plant state value, xk, a feedback control law,uk

k = (xk), is achieved from the open-loop calculation. Because the optimization (2.16)is time-invariant, this feedback law is also time-invariant if the plant model is fixed. Whena PEC is included into the MPC, as the MPC solution looks forward Np steps but im-plements only a single term in the solution, it is necessary to insist that the PEC con-siders the relationship between the applied new control signal, uk

k, and its recent past,{uk−P−N+2

k−P−N+2, . . . ,uk−1k−1}. This does not affect the feedback nature of the control law. But

it does force its dependence on more than the current plant state value. That is, PE-MPCrequires that the state feedback control law has the memory of a certain number of pre-vious inputs. Hence PE-MPC is a dynamic state feedback controller. We formulate thisnext.

Using Definition 4.1.5, a constraint suitable for implementation with PE-MPC maybe derived. The basic idea of the sufficient richness constraint implementation is to spec-ify an additional constraint at time k on the first solution variable uk

k alone. Subsequentlythe superscripts on uk are omitted.

84



θ = {Ai, i = 1, . . . , �; Bi, i = 1, . . . , q; Ci, i = 1, . . . , r}

via

ωk = yk − yk,


+ Crωk−r

].






k,ukk+1, . . . ,u









84



θ = {Ai, i = 1, . . . , �; Bi, i = 1, . . . , q; Ci, i = 1, . . . , r}

via

ωk = yk − yk,


+ Crωk−r

].






k,ukk+1, . . . ,u









84



θ = {Ai, i = 1, . . . , �; Bi, i = 1, . . . , q; Ci, i = 1, . . . , r}

via

ωk = yk − yk,


+ Crωk−r

].






k,ukk+1, . . . ,u









84

Persistently Exciting Model Predictive Control

Define P ≥ N as the backwards-looking input excitation horizon and

Ωk =P−1∑j=0

⎡⎢⎢⎢⎣

uk−j

uk−1−j...

uk−N+1−j

⎤⎥⎥⎥⎦⎡⎢⎢⎢⎣

uk−j

uk−1−j...

uk−N+1−j

⎤⎥⎥⎥⎦T

then write the richness condition (4.18) as

ρ1INm > Ωk > ρ0INm. (4.22)

The focus from here on is solely on the lower bound in (4.22), because the achievementof the upper bound is a requirement of the MPC controller achieving regulation, such asis addressed in many works on MPC using techniques such as terminal state constraintsand horizon choice. We assume that this condition has been met and that the inclusionof an additional lower-bounding excitation constraint to be asserted in PE-MPC does notviolate this. Our focus is on achieving uniform adaptation of plant models via persistenceof excitation. Without assuming adequate bounds on the loop signals, the focus of theadaptation on model consistency is moot.

Thus (4.22) can be written in a more compact form

Ωk =P−1∑j=0

φk−jφTk−j − ρ0INm > 0, (4.23)

where

φi =

⎡⎢⎢⎢⎣

ui

ui−1...

ui−N+1

⎤⎥⎥⎥⎦ . (4.24)

Further, rewritten as

φi =

[ui

φ−i−1

]

where φ−i−1 is a m(N−1)×1 dimensional vector obtained from φi−1 by removing the last

m elements, which by time k have already been determined and therefore are not subjectto the current optimization. With this new notation, (4.23) becomes

Ωk =P−1∑j=0

[uk−j

φ−k−1−j

] [uT

k−j φ−Tk−1−j

]− ρ0INm

=

[uku

Tk +

∑P−1j=1 uk−ju

Tk−j − ρ0Im ukφ

−Tk−1 +

∑P−1j=1 uk−jφ

−Tk−1−j

φ−k−1u

Tk +

∑P−1j=1 φ−

k−1−juTk−j

∑P−1j=0 φ−

k−1−jφ−Tk−1−j − ρ0Im(N−1)

]> 0.

85



Ωk =P−1∑j=0

⎡⎢⎢⎢⎣

uk−j

uk−1−j...

uk−N+1−j

⎤⎥⎥⎥⎦⎡⎢⎢⎢⎣

uk−j

uk−1−j...

uk−N+1−j

⎤⎥⎥⎥⎦T


ρ1INm > Ωk > ρ0INm. (4.22)



Ωk =P−1∑j=0

φk−jφTk−j − ρ0INm > 0, (4.23)

where

φi =

⎡⎢⎢⎢⎣

ui

ui−1...

ui−N+1

⎤⎥⎥⎥⎦ . (4.24)


φi =

[ui

φ−i−1

]



Ωk =P−1∑j=0

[uk−j

φ−k−1−j

] [uT


]− ρ0INm

=

[uku

Tk +

∑P−1j=1 uk−ju


−Tk−1 +


−Tk−1−j

φ−k−1u

Tk +

∑P−1j=1 φ−

k−1−juTk−j

∑P−1j=0 φ−


]> 0.

85



Ωk =P−1∑j=0

⎡⎢⎢⎢⎣

uk−j

uk−1−j...

uk−N+1−j

⎤⎥⎥⎥⎦⎡⎢⎢⎢⎣

uk−j

uk−1−j...

uk−N+1−j

⎤⎥⎥⎥⎦T


ρ1INm > Ωk > ρ0INm. (4.22)



Ωk =P−1∑j=0

φk−jφTk−j − ρ0INm > 0, (4.23)

where

φi =

⎡⎢⎢⎢⎣

ui

ui−1...

ui−N+1

⎤⎥⎥⎥⎦ . (4.24)


φi =

[ui

φ−i−1

]



Ωk =P−1∑j=0

[uk−j

φ−k−1−j

] [uT


]− ρ0INm

=

[uku

Tk +

∑P−1j=1 uk−ju


−Tk−1 +


−Tk−1−j

φ−k−1u

Tk +

∑P−1j=1 φ−

k−1−juTk−j

∑P−1j=0 φ−


]> 0.

85



Ωk =P−1∑j=0

⎡⎢⎢⎢⎣

uk−j

uk−1−j...

uk−N+1−j

⎤⎥⎥⎥⎦⎡⎢⎢⎢⎣

uk−j

uk−1−j...

uk−N+1−j

⎤⎥⎥⎥⎦T


ρ1INm > Ωk > ρ0INm. (4.22)



Ωk =P−1∑j=0

φk−jφTk−j − ρ0INm > 0, (4.23)

where

φi =

⎡⎢⎢⎢⎣

ui

ui−1...

ui−N+1

⎤⎥⎥⎥⎦ . (4.24)


φi =

[ui

φ−i−1

]



Ωk =P−1∑j=0

[uk−j

φ−k−1−j

] [uT


]− ρ0INm

=

[uku

Tk +

∑P−1j=1 uk−ju


−Tk−1 +


−Tk−1−j

φ−k−1u

Tk +

∑P−1j=1 φ−

k−1−juTk−j

∑P−1j=0 φ−


]> 0.

85


Now applying Theorem 4.1.2 we obtain that Ωk > 0 if the previous regressors fulfill

Ω−k−1 =

P−1∑j=0

φ−k−1−jφ

−Tk−1−j − ρ0Im(N−1) > 0 (4.25)

and the Schur complement satisfies

Ωk/Ω−k−1 = uku

Tk +

P−1∑j=1

uk−juTk−j − ρ0Im −

(ukφ

−Tk−1 +

P−1∑j=1

uk−jφ−Tk−1−j

)

(Ω−

k−1

)−1(φ−

k−1uTk +

P−1∑j=1

φ−k−1−ju

Tk−j

)> 0. (4.26)

By noticing that Ω−k−1 is the leading principal submatrix of Ωk−1 − ρ0INm > 0, it is

concluded that (4.25) is always satisfied if (4.23) holds at time k − 1. Thus, after somesimple manipulation of (4.26), the PE candidate constraint for the MPC is

αukuTk + ukβ

T + βuTk + Γ > 0 (4.27)

where

α = 1− φ−Tk−1

(Ω−

k−1

)−1

φ−k−1, (4.28a)

β = −P−1∑j=1

uk−jφ−Tk−1

(Ω−

k−1

)−1

φ−k−1, (4.28b)

Γ =P−1∑j=1


P−1∑j=1


(Ω−

k−1

)−1P−1∑j=1

φ−k−1−ju

Tk−j. (4.28c)

Note that α is a scalar, β is an m× 1 vector, and Γ is an m×m matrix.

Proposition 4.2.1. The MPC candidate PE constraint (4.27) is non-convex.

Proof. Relation (4.23) is equivalent to

Ωk = φkφTk +Ωk−1 − φk−Pφ

Tk−P − ρ0INm > 0. (4.29)

Thus, definingΩ+

k−1 = Ωk−1 − φk−PφTk−P − ρ0INm,

Ωk may be written asΩk = φkφ

Tk + Ω+

k−1 > 0.

86



Ω−k−1 =

P−1∑j=0

φ−k−1−jφ

−Tk−1−j − ρ0Im(N−1) > 0 (4.25)



Tk +

P−1∑j=1


(ukφ

−Tk−1 +

P−1∑j=1


)

(Ω−

k−1

)−1(φ−

k−1uTk +

P−1∑j=1

φ−k−1−ju

Tk−j

)> 0. (4.26)



αukuTk + ukβ

T + βuTk + Γ > 0 (4.27)

where

α = 1− φ−Tk−1

(Ω−

k−1

)−1

φ−k−1, (4.28a)

β = −P−1∑j=1

uk−jφ−Tk−1

(Ω−

k−1

)−1

φ−k−1, (4.28b)

Γ =P−1∑j=1


P−1∑j=1


(Ω−

k−1

)−1P−1∑j=1

φ−k−1−ju

Tk−j. (4.28c)





Tk−P − ρ0INm > 0. (4.29)

Thus, definingΩ+



Tk + Ω+

k−1 > 0.

86



Ω−k−1 =

P−1∑j=0

φ−k−1−jφ

−Tk−1−j − ρ0Im(N−1) > 0 (4.25)



Tk +

P−1∑j=1


(ukφ

−Tk−1 +

P−1∑j=1


)

(Ω−

k−1

)−1(φ−

k−1uTk +

P−1∑j=1

φ−k−1−ju

Tk−j

)> 0. (4.26)



αukuTk + ukβ

T + βuTk + Γ > 0 (4.27)

where

α = 1− φ−Tk−1

(Ω−

k−1

)−1

φ−k−1, (4.28a)

β = −P−1∑j=1

uk−jφ−Tk−1

(Ω−

k−1

)−1

φ−k−1, (4.28b)

Γ =P−1∑j=1


P−1∑j=1


(Ω−

k−1

)−1P−1∑j=1

φ−k−1−ju

Tk−j. (4.28c)





Tk−P − ρ0INm > 0. (4.29)

Thus, definingΩ+



Tk + Ω+

k−1 > 0.

86



Ω−k−1 =

P−1∑j=0

φ−k−1−jφ

−Tk−1−j − ρ0Im(N−1) > 0 (4.25)



Tk +

P−1∑j=1


(ukφ

−Tk−1 +

P−1∑j=1


)

(Ω−

k−1

)−1(φ−

k−1uTk +

P−1∑j=1

φ−k−1−ju

Tk−j

)> 0. (4.26)



αukuTk + ukβ

T + βuTk + Γ > 0 (4.27)

where

α = 1− φ−Tk−1

(Ω−

k−1

)−1

φ−k−1, (4.28a)

β = −P−1∑j=1

uk−jφ−Tk−1

(Ω−

k−1

)−1

φ−k−1, (4.28b)

Γ =P−1∑j=1


P−1∑j=1


(Ω−

k−1

)−1P−1∑j=1

φ−k−1−ju

Tk−j. (4.28c)





Tk−P − ρ0INm > 0. (4.29)

Thus, definingΩ+



Tk + Ω+

k−1 > 0.

86


From the matrix determinant lemma, we have that, for an invertible matrix M , and twocolumn vectors v1, v2

det(M + v1vT2 ) = (1 + vT

2 M−1v1) det(M ).

Applying it to Ωk and assuming that det(Ω+k−1) �= 0, we obtain

det(Ωk) = det(Ω+k−1 + φkφ

Tk ) = (1 + φT

k (Ω+k−1)

−1φk) det(Ω+k−1) > 0.

Now, let us distinguish three cases:Ω+

k−1 < 0 =⇒ det(Ω+k−1) < 0, thus to have det(Ωk) > 0, we need 1 +

φTk (Ω

+k−1)

−1φk < 0 which implies that uk must be outside the ellipsoid

φTk (Ω

+k−1)

−1φk > 1.

Ω+k−1 > 0 =⇒ det(Ω+

k−1) > 0, thus to have det(Ωk) > 0, we need 1 +

φTk (Ω

+k−1)

−1φk > 0 which is always satisfied. This could happen ifP > N , and the previous input sequence excitation level is such that,adding any new input to the sequence at time k, will still conserve theminimum excitation requirement.

Ω+k−1 is indefinite then it will be still non-convex but in a lower dimension.

Remarks:• Note that (4.27) defines the non-convex exterior of an ellipsoid.• By using the Schur complement the PEC matrix (4.22) dimensions are reduced,

from Nm×Nm to m×m, (4.27), with obvious benefits on its implementation.

4.2.2 Persistently Exciting Model Predictive Control formulationThe Persistently Exciting Model Predictive Control formulation that is proposed here isobtained by augmenting the constraint set of (2.16) with the PE constraint (4.27).


minuk

J(k, u) = F (xk+Np) +

Np−1∑j=0



(4.30b)

ukk+j ∈ U , j = 0, . . . , Np − 1,

(4.30c)

xk+j ∈ X , j = 0, . . . , Np, (4.30d)

αukuTk + ukβ

T + βuTk + Γ > 0, (4.30e)

87



det(M + v1vT2 ) = (1 + vT

2 M−1v1) det(M ).



Tk ) = (1 + φT

k (Ω+k−1)

−1φk) det(Ω+k−1) > 0.



φTk (Ω

+k−1)


φTk (Ω

+k−1)

−1φk > 1.

Ω+k−1 > 0 =⇒ det(Ω+


φTk (Ω

+k−1)







minuk

J(k, u) = F (xk+Np) +

Np−1∑j=0



(4.30b)

ukk+j ∈ U , j = 0, . . . , Np − 1,

(4.30c)

xk+j ∈ X , j = 0, . . . , Np, (4.30d)

αukuTk + ukβ

T + βuTk + Γ > 0, (4.30e)

87



det(M + v1vT2 ) = (1 + vT

2 M−1v1) det(M ).



Tk ) = (1 + φT

k (Ω+k−1)

−1φk) det(Ω+k−1) > 0.



φTk (Ω

+k−1)


φTk (Ω

+k−1)

−1φk > 1.

Ω+k−1 > 0 =⇒ det(Ω+


φTk (Ω

+k−1)







minuk

J(k, u) = F (xk+Np) +

Np−1∑j=0



(4.30b)

ukk+j ∈ U , j = 0, . . . , Np − 1,

(4.30c)

xk+j ∈ X , j = 0, . . . , Np, (4.30d)

αukuTk + ukβ

T + βuTk + Γ > 0, (4.30e)

87



det(M + v1vT2 ) = (1 + vT

2 M−1v1) det(M ).



Tk ) = (1 + φT

k (Ω+k−1)

−1φk) det(Ω+k−1) > 0.



φTk (Ω

+k−1)


φTk (Ω

+k−1)

−1φk > 1.

Ω+k−1 > 0 =⇒ det(Ω+


φTk (Ω

+k−1)







minuk

J(k, u) = F (xk+Np) +

Np−1∑j=0



(4.30b)

ukk+j ∈ U , j = 0, . . . , Np − 1,

(4.30c)

xk+j ∈ X , j = 0, . . . , Np, (4.30d)

αukuTk + ukβ

T + βuTk + Γ > 0, (4.30e)

87


where α, β and Γ are defined in (4.28) and are functions of previous inputs only.Note that the PE constraint (4.30e) is a quadratic function of only the first input

uk. The coefficients of this constraint, α, β, and Γ, are computed every time step andare time-varying. The constrained optimization problem (4.30) is a standard NonlinearModel Predictive Control (NMPC) formulation as for example defined in Mayne [2000].The additional constraint (4.27) or (4.30e) is fundamental to preserving a minimal levelof excitation for the identification and adaptation of the system. We next explore some ofits properties.

Theorem 4.2.2. Consider the PE-MPC problem (4.30) at time k with input-only con-straints, i.e. X = Rn. Assume that the solutions at times k−1 through k−P −N+1 arefeasible and that the resulting PE-MPC solutions yield strictly bounded plant state xk.Then the solution uk = uk−P at time k is feasible, although not necessarily optimal, andits repeated application for all k is feasible and yields a P -periodic solution for {uk}.

Proof. At time k − 1 a feasible solution is {uk−1k−1,u

k−1k , . . . ,uk−1

k+Np−2}. This indicatesthe input constraint (4.30c) is satisfied. Moreover, (4.30e) and the PEC (4.23), thus theconstraint (4.30e) is derived and are both satisfied.

At time k, due to the time-varying nature of the constraint (4.27), it is necessary toguarantee that the solution {uk

k,ukk+1, . . . ,u

kk+Np−1} is still feasible.

Recalling that (4.30e) is equivalent to (4.29) and considering that Ωk−1 − ρ0INm >0, since it is the PEC at time k − 1, then (4.30e) (or equivalently (4.29)) is satisfied ifφkφ

Tk − φk−Pφ

Tk−P ≥ 0.

Recalling (4.24), and that the constraint (4.30e) affects only the first input uk, it isclear that choosing uk = uk−P in φk results in a feasible solution at time k. Obviously,uk = uk−P also satisfies the input constraint.

Finally, note that due to the receding horizon property, only the first input uk of thesequence {uk

k,ukk+1, . . . ,u

kk+Np−1} is applied. This, would naturally yield a P -periodic

input sequence if exactly this input was repeatedly chosen.

Corollary 4.2.3. Consider the PE-MPC problem of Theorem 4.2.2 with feasible periodicinput uk = uk−P in the scalar (m = 1) case, then this input sequence is the sum of atleast N complex sinusoids with non-zero amplitudes and different frequencies.

Proof. Since {uk} is feasible and periodic, the regressor vector φk is also periodic andhas associated N×N matrix Ωk from (4.22) fixed and positive definite. The result followsby appealing to the frequency domain interpretation of the SRC (4.18) from Goodwin &Sin [1984].

88






k−1k , . . . ,uk−1



k,ukk+1, . . . ,u



Tk − φk−Pφ

Tk−P ≥ 0.



k,ukk+1, . . . ,u





88






k−1k , . . . ,uk−1



k,ukk+1, . . . ,u



Tk − φk−Pφ

Tk−P ≥ 0.



k,ukk+1, . . . ,u





88






k−1k , . . . ,uk−1



k,ukk+1, . . . ,u



Tk − φk−Pφ

Tk−P ≥ 0.



k,ukk+1, . . . ,u





88

Examples using Finite Impulse Response models

Remarks:• The dual adaptive control effect is evident in the PE-MPC formulation. While the

standard MPC seeks to drive the input uk to zero, stabilizing the plant, the PEconstraint (4.27) tries to bound uk away from zero. Indeed, the cost of persistenceof excitation is seen explicitly in the inclusion of an additional constraint (4.30e).

• The periodic solution uk = uk−P is feasible, but not necessarily optimal.• In the case where state constraints are considered, their satisfaction would need to

be separately ensured to establish feasibility of the PE-MPC problem.• In time-invariant situations where the MPC is able to stabilize the system to the

strict interior of the state-constraint set, X , and where the requisite excitation level,ρ0, is correspondingly modest, one would presume that the periodic solution wouldbe the asymptotic limit of the control signal. This has yet to be proven. Although,this is the case in the following example.

• The periodic solution arises directly from the PEC formulation. Thus, there is noneed to force it by an equality constraint as done in Genceli & Nikolaou [1996].

4.3 Examples using Finite Impulse Response models

This and subsequent sections give several examples to show how the PE-MPC can in-herently produce persistently exciting inputs and tend towards periodic solutions. Someinteresting properties of the proposed approach are discussed.

In the first example a plant is regulated by a constrained predictive controller basedon an FIR model and an RLS algorithm is used to continually estimate the Markov pa-rameters of the plant. The coefficients of the FIR model used in the MPC are updatedevery 50 time steps.

The plant is itself described by an FIR model and so is open-loop stable;

yk = φkθT + vk,

where {vk} is a Gaussian measurement noise sequence with covariance σv = 6.25 · 10−4,N = 3, the regressor is φk =

[uk uk−1 uk−2

]T , and

θ =

⎡⎣ 0.03

0.280.63

⎤⎦

are the ‘true’ system parameters.In this case, for a scalar input, starting from the FIR based MPC formulation (2.13),

89












yk = φkθT + vk,


[uk uk−1 uk−2

]T , and

θ =

⎡⎣ 0.03

0.280.63

⎤⎦


89












yk = φkθT + vk,


[uk uk−1 uk−2

]T , and

θ =

⎡⎣ 0.03

0.280.63

⎤⎦


89












yk = φkθT + vk,


[uk uk−1 uk−2

]T , and

θ =

⎡⎣ 0.03

0.280.63

⎤⎦


89


the PE-MPC formulation becomes

minu

Jk =1

2

Np−1∑j=0

‖ yk+j+1 ‖2Q + ‖ uk+j ‖2R, (4.31a)


umin ≤ uk+j ≤ umax, j = 0, . . . , Np − 1, (4.31c)αu2

k + 2βuk + γ > 0. (4.31d)

with prediction horizon Np = 10, input constraints −5 ≤ u ≤ 5, cost function outputweight Q = 5, and input weight R = 0.3. The time varying scalars (α, β, γ), from the PEconstraint (4.31d), are computed every time step as in (4.28), and the backwards-lookingexcitation horizon length is chosen as P = 7 which exceeds N = 3, and the SRC designparameter is selected as ρ0 = 2.5.

Decomposing the quadratic constraint (4.31d) into two linear constraints, it is pos-sible to formulate the FIR model MPC as two Quadratic Programming (QP) problems.Thus, the optimal solution is obtained by choosing the better solution between the twoQP problems.

A standard RLS algorithm without the forgetting factor, and initial conditions

θ0 =

⎡⎣ 0

00

⎤⎦ , P0 =

⎡⎣ 1000 0 0

0 1000 00 0 1000

⎤⎦ ,

is used to adapt the model parameters. The estimation algorithm is not reset but the MPChas its model updated every 50 samples.

In the second example, to allow set-point changes, the cost function (4.31a) is modi-fied as follows

Jk =1

2

Np−1∑j=0

‖ yk+j+1 − ys ‖2Q + ‖ Δuk+j ‖2R, (4.32)

where ys is the set-point and Δuj = uj − uj−1.Finally in both cases the MPC model is initialized with the following FIR coefficients

θ0 =

⎡⎣ 0.13

01.26

⎤⎦ .

90



minu

Jk =1

2

Np−1∑j=0

‖ yk+j+1 ‖2Q + ‖ uk+j ‖2R, (4.31a)



k + 2βuk + γ > 0. (4.31d)




θ0 =

⎡⎣ 0

00

⎤⎦ , P0 =

⎡⎣ 1000 0 0

0 1000 00 0 1000

⎤⎦ ,



Jk =1

2

Np−1∑j=0

‖ yk+j+1 − ys ‖2Q + ‖ Δuk+j ‖2R, (4.32)


θ0 =

⎡⎣ 0.13

01.26

⎤⎦ .

90



minu

Jk =1

2

Np−1∑j=0

‖ yk+j+1 ‖2Q + ‖ uk+j ‖2R, (4.31a)



k + 2βuk + γ > 0. (4.31d)




θ0 =

⎡⎣ 0

00

⎤⎦ , P0 =

⎡⎣ 1000 0 0

0 1000 00 0 1000

⎤⎦ ,



Jk =1

2

Np−1∑j=0

‖ yk+j+1 − ys ‖2Q + ‖ Δuk+j ‖2R, (4.32)


θ0 =

⎡⎣ 0.13

01.26

⎤⎦ .

90



minu

Jk =1

2

Np−1∑j=0

‖ yk+j+1 ‖2Q + ‖ uk+j ‖2R, (4.31a)



k + 2βuk + γ > 0. (4.31d)




θ0 =

⎡⎣ 0

00

⎤⎦ , P0 =

⎡⎣ 1000 0 0

0 1000 00 0 1000

⎤⎦ ,



Jk =1

2

Np−1∑j=0

‖ yk+j+1 − ys ‖2Q + ‖ Δuk+j ‖2R, (4.32)


θ0 =

⎡⎣ 0.13

01.26

⎤⎦ .

90


4.3.1 Simulation resultsThe simulations are run in MATLAB 7.9 using the solver e04nf from the NAG libraryto solve the QP problems every time step. For every simulation, in the first P + 1 timesteps standard FIR MPC (without PE constraint) is used and it is also checked that all thenecessary vectors for the PE constraint formulation are properly initialized.

Regulation to zero

� ��

�

��

�

��

�

��

��

� ��

��

�

��

�

��

��!��"��

Figure 4.1: Plant input and output signals.

Figure 4.1 shows the plant input and output. Input magnitude constraints are neveractive and the input signal is effectively driven entirely by the SRC/PEC constraint. InFigure 4.2 the FIR coefficients are shown, the dashed lines represent the real parametervalues θ0, the solid lines are the RLS estimates, and the stars are the occasionally updatedvalues used in the MPC model. For example, the stars at time step 50 are the FIR co-efficients θ, they are used in the controller for the first 50 steps, then the estimate fromRLS is used to update the MPC model, which is again kept constant until the next updateoccurs (50 steps later).

Finally, Figure 4.3 shows the plant input in the time domain, and more importantly

91



Regulation to zero

� ��

�

��

�

��

�

��

��

� ��

��

�

��

�

��

��!��"��




91



Regulation to zero

� ��

�

��

�

��

�

��

��

� ��

��

�

��

�

��

��!��"��




91



Regulation to zero

� ��

�

��

�

��

�

��

��

� ��

��

�

��

�

��

��!��"��




91


0 50 100 150 200 250 300 350 400 450 5000

0.05

0.1

0.15

0.2

1

FIR model coefficients

0 50 100 150 200 250 300 350 400 450 5000

0.1

0.2

0.3

0.4

2

0 50 100 150 200 250 300 350 400 450 5000

0.5

1

1.5

3

Time [s], Sampling Time: 1

Figure 4.2: FIR parameters: RLS estimates (solid line), ‘real’ values (dashed line), MPCmodel parameter (stars).

� ��

�

��

�

��

�

��#$��#%&$

��'��%��# ��

&�$�

��

��

��

��

��%��$&��'%�$(&��)��&�$�

�(*&#�+��-�

;��

��;

Figure 4.3: Persistently exciting input and its spectrum indicating excitation suited to thefitting of a three-parameter model.

92


0 50 100 150 200 250 300 350 400 450 5000

0.05

0.1

0.15

0.2

1


0 50 100 150 200 250 300 350 400 450 5000

0.1

0.2

0.3

0.4

2

0 50 100 150 200 250 300 350 400 450 5000

0.5

1

1.5

3



� ��

�

��

�

��

�

��#$��#%&$

��'��%��# ��

&�$�

��

��

��

��

��%��$&��'%�$(&��)��&�$�

�(*&#�+��-�

;��

��;


92


0 50 100 150 200 250 300 350 400 450 5000

0.05

0.1

0.15

0.2

1


0 50 100 150 200 250 300 350 400 450 5000

0.1

0.2

0.3

0.4

2

0 50 100 150 200 250 300 350 400 450 5000

0.5

1

1.5

3



� ��

�

��

�

��

�

��#$��#%&$

��'��%��# ��

&�$�

��

��

��

��

��%��$&��'%�$(&��)��&�$�

�(*&#�+��-�

;��

��;


92


0 50 100 150 200 250 300 350 400 450 5000

0.05

0.1

0.15

0.2

1


0 50 100 150 200 250 300 350 400 450 5000

0.1

0.2

0.3

0.4

2

0 50 100 150 200 250 300 350 400 450 5000

0.5

1

1.5

3



� ��

�

��

�

��

�

��#$��#%&$

��'��%��# ��

&�$�

��

��

��

��

��%��$&��'%�$(&��)��&�$�

�(*&#�+��-�

;��

��;


92


the magnitude of its Fourier transform. The three peaks confirm that the input is per-sistently exciting of order three, which is the minimum order needed to estimate threeparameters. This figure also confirms that uk is tending to become periodic, as expected.

Variable output set-point

A more complex scenario is shown here, where at steps k = 75 and k = 200 set-pointchanges are given. In addition, at step k = 125 a plant change is simulated with the newplant coefficients being

θ =

⎡⎣ 0.5

0.20.3

⎤⎦ .

The simulation parameters are the same as in the previous case with the exception of thecost function output weight Q = 1, the weight on the input difference (Δu) R = 3,the backwards-looking excitation horizon length P = 4, and the SRC design parameterρ0 = 4.9.

Figure 4.4 shows plant input and output signals. It is evident that the PE-MPC con-troller is able to follow the output set-point (red-dashed line). The oscillation along theset-point is due to the effect of the PE-constraint, that is expected and needed to producean SRC signal for estimating the three FIR parameters. As result in Figure 4.5 the esti-mated parameters converge to the true values, even when at step k = 125 a change in theplant occurs. Note that at step k = 150 the speed of converge suddenly increases, this isdue to the covariance resetting into the RLS estimate algorithm that is performed everytime the MPC model is updated. To confirm that the input signal is sufficiently rich, theamplitude spectrum is plotted in Figure 4.6.

Finally, Figure 4.7 shows an instance of how PE-MPC is divided into two QP prob-lems, and how the PE constraints are positioned with respect to the bound constraints onthe input. It is possible to note the dual effect introduced by the PE constraints. While thestandard MPC would try to drive the input to zero, the new constraint bounds the inputaway from zero. This produces a small performance degradation, but it is necessary to ob-tain the dual control feature such that the controller is able to produce sufficient excitationto learn from the plant (estimate its parameters). The PE-MPC optimal solution is chosenbetween the two QP optimal solutions, and when one is not feasible it is chosen from theremaining one. Note that at no point both QPs are infeasible. If both QPs are infeasibleat the same time, this would indicate a too high value of the SRC design parameter ρ0.

93





θ =

⎡⎣ 0.5

0.20.3

⎤⎦ .




93





θ =

⎡⎣ 0.5

0.20.3

⎤⎦ .




93





θ =

⎡⎣ 0.5

0.20.3

⎤⎦ .




93


� ��

�

�

�

�

��

��

� ��

�

�

�

�

�

�

��

��!��"��

Figure 4.4: Plant input and output signals. The dashed line indicates the set-point.

� ��

�

��

��

��

�

��

� ��

��

��

��

��

�

� ��

��

�

��

�

�� !� �� "��


94


� ��

�

�

�

�

��

��

� ��

�

�

�

�

�

�

��

��!��"��


� ��

�

��

��

��

�

��

� ��

��

��

��

��

�

� ��

��

�

��

�

�� !� �� "��


94


� ��

�

�

�

�

��

��

� ��

�

�

�

�

�

�

��

��!��"��


� ��

�

��

��

��

�

��

� ��

��

��

��

��

�

� ��

��

�

��

�

�� !� �� "��


94


� ��

�

�

�

�

��

��

� ��

�

�

�

�

�

�

��

��!��"��


� ��

�

��

��

��

�

��

� ��

��

��

��

��

�

� ��

��

�

��

�

�� !� �� "��


94


��

��

��

��

��

�

��

��

��

�!

��

Figure 4.6: Spectrum of the input indicating excitation suited to the fitting of a three-parameter model.

��

�

�

�

�

�

�

��

�� "�

!� ��"��#��

$��%��!� ��"��#��

&��%��'�<��(�%�)

&��%��*��(�%�)

��

!�

!� �

Figure 4.7: Representation of PE-MPC constraints and optimal solution for a short simu-lation interval.

95


��

��

��

��

��+

�

��

��

��

�!

��


��

�

�

�

�

�

�

��

�� "�

!� ��"��#��

$��%��!� ��"��#��

&��%��'�<��(�%�)

&��%��*��(�%�)

��

!�

!� �


95


��

��

��

��

��+

�

��

��

��

�!

��


��

�

�

�

�

�

�

��

�� "�

!� ��"��#��

$��%��!� ��"��#��

&��%��'�<��(�%�)

&��%��*��(�%�)

��

!�

!� �


95


��

��

��

��

��+

�

��

��

��

�!

��


��

�

�

�

�

�

�

��

�� "�

!� ��"��#��

$��%��!� ��"��#��

&��%��'�<��(�%�)

&��%��*��(�%�)

��

!�

!� �


95


Next, it is shown how the choice of certain design parameters such as the backwards-looking excitation horizon P and the SRC design parameter ρ0 affects the excitation leveland thus the speed of convergence of the FIR estimates. Figures 4.8 and 4.9 are obtainedwith a zero set-point, ρ0 = 2.5, P = 3, and are used as base for comparison. Note that atk = 55 a plant change occurs. When a longer excitation horizon P = 5 is used, the inputchanges shape (Figure 4.10) and this has a direct effect by slowing the estimate speedof convergence as shown in Figure 4.11. When instead a smaller ρ0 = 1.8 is chosen(with P = 3), the input keeps almost the same shape (Figure 4.12), but with a reducedexcitation level the FIR parameter estimates converge slowly (Figure 4.13).

� ��

�

��

��

� ��

�

�

�

�

��

��

Figure 4.8: Plant input and output signals, base example for comparison. The dashed lineindicates the set-point.

96



� ��

�

��

��

� ��

�

�

�

�

��

��


96



� ��

�

��

��

� ��

�

�

�

�

��

��


96



� ��

�

��

��

� ��

�

�

�

�

��

��


96


0 10 20 30 40 50 60 70 80 90 1000

0.2

0.4

0.6

0.8

1


0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

2

0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

3


Figure 4.9: FIR parameters, base example for comparison: RLS estimates (solid line),‘real’ values (dashed line), MPC model parameter (stars).

� ��

�

��

��

� ��

�

�

��

��

Figure 4.10: Plant input and output signals when using longer excitation horizon P . Thedashed line indicates the set-point.

97


0 10 20 30 40 50 60 70 80 90 1000

0.2

0.4

0.6

0.8

1


0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

2

0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

3



� ��

�

��

��

� ��

�

�

��

��


97


0 10 20 30 40 50 60 70 80 90 1000

0.2

0.4

0.6

0.8

1


0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

2

0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

3



� ��

�

��

��

� ��

�

�

��

��


97


0 10 20 30 40 50 60 70 80 90 1000

0.2

0.4

0.6

0.8

1


0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

2

0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

3



� ��

�

��

��

� ��

�

�

��

��


97


! �! �! �! �! �! ! ��

�

��

��

��

�

��

� ��

��

��

��

��

�

� ��

��

�

��

�

�� !��"��#��

Figure 4.11: FIR parameters when using longer excitation horizon P : RLS estimates(solid line), ‘real’ values (dashed line), MPC model parameter (stars).

� ��

�

��

��

� ��

�

�

�

�

�

�

��

��

Figure 4.12: Plant input and output signals when using smaller SRC design parameter ρ0.The dashed line indicates the set-point.

98


� ��

�

��

��

��

�

��

� ��

��

��

��

��

�

� ��

��

�

��

�

�� !��"��#��


� ��

�

��

��

� ��

�

�

�

�

�

�

��

��


98


� ��

�

��

��

��

�

��

� ��

��

��

��

��

�

� ��

��

�

��

�

�� !��"��#��


� ��

�

��

��

� ��

�

�

�

�

�

�

��

��


98


� ��

�

��

��

��

�

��

� ��

��

��

��

��

�

� ��

��

�

��

�

�� !��"��#��


� ��

�

��

��

� ��

�

�

�

�

�

�

��

��


98


0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1


0 10 20 30 40 50 60 70 80 90 1000

0.2

0.4

2

0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

3


Figure 4.13: FIR parameters when using smaller SRC design parameter ρ0: RLS estimates(solid line), ‘real’ values (dashed line), MPC model parameter (stars).

99


0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1


0 10 20 30 40 50 60 70 80 90 1000

0.2

0.4

2

0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

3



99


0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1


0 10 20 30 40 50 60 70 80 90 1000

0.2

0.4

2

0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

3



99


0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1


0 10 20 30 40 50 60 70 80 90 1000

0.2

0.4

2

0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

3



99


4.4 Examples using state space modelsThis section describes two examples of an PE-MPC controller based on state space mod-els. The plant is itself described by the following state space model

xk+1 = Axk +Buk + wk

yk = xk + vk,

with A = 1 and B = 0.5, and where {wk}, and {vk} are Gaussian process and measure-ment noise sequences with zero means and covariances σw = 1·10−4 and σv = 6.25·10−4,respectively.

The PE-MPC formulation is

minu

Jk =1

2

Np−1∑j=0

‖ ys − yk+j+1 ‖2Q + ‖ Δuk+j ‖2R, (4.33a)

s.t. xk+1+j = Axk+j + Buk+j, j = 1, . . . , Np, (4.33b)


k + 2βuk + γ > 0. (4.33d)

with Δuj = uj − uj−1, prediction horizon Np = 10, input constraints −0.1 ≤ u ≤ 0.1,cost function output weight Q = 0.1, and weight on the input changes R = 1000. Thetime varying scalars (α, β, γ), from the PE constraint (4.33d), are computed every timestep as in (4.28), and the backwards-looking excitation horizon length is chosen as P = 18which exceeds N = 2, finally the SRC design parameter is selected as ρ0 = 0.01.

An RLS algorithm is implemented to estimate the parameter of the following ARMAmodel

yk = ayk−1 + buk−1

where

θk =

[ab

]=

[−A

B

], (4.34)

and the associated regressor is

φ(k) =[yk−1 uk−1

]T. (4.35)

It is important to note that the regressor contains past inputs and outputs, thus Corollary4.1.4 must be verified. In this case this is done trivially, thus a persistently exciting outputmay be obtained by a sufficiently rich input.

100




yk = xk + vk,



minu

Jk =1

2

Np−1∑j=0

‖ ys − yk+j+1 ‖2Q + ‖ Δuk+j ‖2R, (4.33a)



k + 2βuk + γ > 0. (4.33d)




where

θk =

[ab

]=

[−A

B

], (4.34)


φ(k) =[yk−1 uk−1

]T. (4.35)


100




yk = xk + vk,



minu

Jk =1

2

Np−1∑j=0

‖ ys − yk+j+1 ‖2Q + ‖ Δuk+j ‖2R, (4.33a)



k + 2βuk + γ > 0. (4.33d)




where

θk =

[ab

]=

[−A

B

], (4.34)


φ(k) =[yk−1 uk−1

]T. (4.35)


100




yk = xk + vk,



minu

Jk =1

2

Np−1∑j=0

‖ ys − yk+j+1 ‖2Q + ‖ Δuk+j ‖2R, (4.33a)



k + 2βuk + γ > 0. (4.33d)




where

θk =

[ab

]=

[−A

B

], (4.34)


φ(k) =[yk−1 uk−1

]T. (4.35)


100

Examples using state space models

4.4.1 Simulation resultsThe first simulation (Figure 4.14(a)) shows that starting from zero initial condition, theoutput is driven to ys = 1 and then at k = 200 a set-point change occurs. The modelparameter convergence and MPC model parameter updates are shown in Figure 4.14(b).

It is important to note in Figure 4.15 that due to the difference between the zeroinitial condition and the output set-point ys = 1, both PE constraints are not active. Infact only one QP problem (green color in Figure 4.15(b)) is solved every time step for thefirst part. Then when the actual plant output gets close to the set-point the excitation leveldiminishes making the PE constraints active and yielding two distinct QP problems (blueand red Figure 4.15(b)). This is interesting behavior because it shows that when, for someexternal reason, the input is already sufficiently rich the PE constraints are not active andthus only one QP problem (corresponding to a standard MPC implementation) has to besolved.

While in the first example full state feedback was assumed, in the next example astandard linear Kalman Filter is used for state estimation. The steady state Kalman gainis re-calculated every time the model is updated (each 50 steps). Figures 4.16(a) and4.16(b) show input, output and model parameters, respectively. This example illustratesthat starting from zero steady state, once the PE-MPC is activated, it drives the plantaway from the steady state, due to the lack of excitation. At step k = 200 a sinusoidalset-point is applied which makes both PE constraints become inactive as given in Figure4.17. Finally, Figure 4.18(a) shows that input and output have two peaks respectively ontheir amplitude spectrum, which confirms the minimum frequency content for estimatingN = 2 parameters. Figure 4.18(b) compares the Kalman state estimate with the real state.

101





101





101





101


� ��

��

�

��

��

��

� ��

�

��

�

��

�

��

��

��!��"��

(a) Plant input and output signals. The dashed line indicates the set-point.

� ��

��

�

��

��

��

��

�

�

��

� ��

��

�

��

��

��

��

�

�

��!��"��

(b) ARMA parameters: RLS estimates (solid line), ‘real’ values (dashedline), MPC model parameter (stars).

Figure 4.14: Input, output and parameters for state space case.

102


� ��

��

�

��

��

��

� ��

�

��

�

��

�

��

��

��!��"��


� ��

��

�

��

��

��

��

�

�

��

� ��

��

�

��

��

��

��

�

�

��!��"��



102


� ��

��

�

��

��

��

� ��

�

��

�

��

�

��

��

��!��"��


� ��

��

�

��

��

��

��

�

�

��

� ��

��

�

��

��

��

��

�

�

��!��"��



102


� ��

��

�

��

��

��

� ��

�

��

�

��

�

��

��

��!��"��


� ��

��

�

��

��

��

��

�

�

��

� ��

��

�

��

��

��

��

�

�

��!��"��



102


� ��

��

��

��

��

��

�

��

��

��

��

��

��

� ��

(a) Representation of PE-MPC constraints and optimal solution.

0 50 100 150 200 250 300 350 4000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1Solved optimization problems


(b) Solved QP problems: QP with no PE constraints (green), QP with upperPE constraint (blue), QP with lower PE constraint (red). Note that the colorindicates which QP is solved.

Figure 4.15: PE-MPC constraints and solved QP problems for state space case.

103


� ��

��

��

��

��

��

�

��

��

��

��

��

��

� ��


0 50 100 150 200 250 300 350 4000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9





103


� ��

��

��

��

��

��

�

��

��

��

��

��

��

� ��


0 50 100 150 200 250 300 350 4000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9





103


� ��

��

��

��

��

��

�

��

��

��

��

��

��

� ��


0 50 100 150 200 250 300 350 4000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9





103


� ��

��

�

��

��

��

� ��

��

��

��

�

��

��

��

��

��


� ��

�

��

�

��

�

��

� ��

��

��

��

��

�

��

��

�

��!�!�� "��


Figure 4.16: Input, output and parameters for state space case with Kalman Filter stateestimate.

104


� ��

��

�

��

��" ��

��

� ��

��

��

��

�

��

��

��

��

��


� ��

�

��

�

��

�

��

� ��

��

��

��

��

�

��

��

�

��!�!�� "��



104


� ��

��

�

��

��" ��

��

� ��

��

��

��

�

��

��

��

��

��


� ��

�

��

�

��

�

��

� ��

��

��

��

��

�

��

��

�

��!�!�� "��



104


� ��

��

�

��

��" ��

��

� ��

��

��

��

�

��

��

��

��

��


� ��

�

��

�

��

�

��

� ��

��

��

��

��

�

��

��

�

��!�!�� "��



104


� ��

��

��

��

��

��

�

��

��

��

��

��

��

� ��!��


0 50 100 150 200 250 300 350 4000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9




Figure 4.17: PE-MPC constraints and solved QP problems for state space case withKalman Filter state estimate.

105


� ��

��

��

��

��

��

�

��

��

��

��

��

��

� ��!��


0 50 100 150 200 250 300 350 4000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9





105


� ��

��

��

��

��

��

�

��

��

��

��

��

��

� ��!��


0 50 100 150 200 250 300 350 4000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9





105


� ��

��

��

��

��

��

�

��

��

��

��

��

��

� ��!��


0 50 100 150 200 250 300 350 4000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9





105


��

��

��

��

��

��

��

��

��

� �

��

�

��

(a) Input and output spectrum.

� ��

��

��

��

�

��

��

��

��

��!��

! � ��

�

�! � ��

(b) State and its estimate.

Figure 4.18: Input and output spectrum, and state and its estimate for state space casewith Kalman Filter state estimate.

106


��

��

��

��

��

��

��

��

��

��

��

�

��


� ��

��

��

��

�

��

��

��

��

��!��

! � ��

�

�! � ��



106


��

��

��

��

��

��

��

��

��

��

��

�

��


� ��

��

��

��

�

��

��

��

��

��!��

! � ��

�

�! � ��



106


��

��

��

��

��

��

��

��

��

��

��

�

��


� ��

��

��

��

�

��

��

��

��

��!��

! � ��

�

�! � ��



106

Conclusions

4.5 ConclusionsA simple modification of the model predictive control formulation is given which yieldspersistently exciting regressor vectors suited to online model adaptation or tuning. Its ef-fectiveness is shown by several computational examples. The approach chosen is similarto the one in Genceli & Nikolaou [1996], but with the difference that the PE constraint isimposed only on the first MPC manipulated variable and via a backwards-looking addi-tional constraint. This alters the MPC controller from memoryless full-state feedback todynamic full-state feedback. This novel framework allowed us to show formally and byexamples that a periodic control signal is feasible and arises autonomously, avoiding thenecessity of explicitly forcing this by adding an equality constraint. Several simulationexamples, for FIR and state space models, with full state and also output feedback aregiven. In all examples it is possible to note that the particular structure of the PE con-straint with scalar inputs gives PE-MPCs that are expressed as two QP problems. In gen-eral, the PE-MPC formulation given is valid for MIMO systems and for general adaptiveschemes, yielding a non-convex optimization problem. In this case general non-convexsolvers may be required or, alternatively, the non-convex SRC constraint (4.30e) mightbe broken down into the union of a number of convex constraints. Efficient numericalimplementation for MIMO systems will be a topic for future research.

107

Conclusions


107

Conclusions


107

Conclusions


107


108


108


108


108

Chapter 5

State estimation issues

5.1 The state estimation problem

The state estimation problem can generally be seen as a probabilistic inference problem,which means how to estimate hidden variables (state) using a noisy set of observations(measurement) in an optimal way. In probability theory, this solution is obtained using therecursive Bayesian estimation algorithm. For linear Gaussian systems, it can be shownthat the closed-form of the optimal recursive solution is given by the well known KalmanFilter.

For real world applications, which are neither linear nor Gaussian, the Bayesian ap-proach is intractable. Thus, a sub-optimal approximated solution has to be found. Forinstance, in the case of a nonlinear system with non-Gaussian probability density func-tion (pdf), the Extended Kalman Filter (EKF) algorithm approximates a nonlinear systemwith its truncated Taylor series expansion around the current estimate. Here, the non-Gaussian pdf is approximated with its first two moments, mean and covariance. A moreelaborate approach is to use a Sequential Monte-Carlo method, where instead of approxi-mating the system or the probability distribution, the integrals in the Bayesian solution areapproximated by finite sums. Although a famous Monte-Carlo filter is the Particle Filter,in this chapter the focus is on Kalman filtering for two simple reasons. Firstly, Kalmanalgorithms are easy to understand and to implement. Secondly, they do not have highcomputational complexity, making them suitable to be coupled with a model predictivecontroller. This is a benefit for real-time implementation. Since all computation must bedone within a maximum time interval, having a quick estimation algorithm allows the useof the remaining time to be used for solving the control optimization problem.

The focus is mainly on Kalman filtering for nonlinear systems. In detail, the Ex-tended Kalman Filter (EKF) and the Unscented Kalman Filter (UKF) are described andtheir performances are compared in two specific cases.

109

Chapter 5






109

Chapter 5






109

Chapter 5






109

5. State estimation issues

5.1.1 Definitions of ObservabilityWhen state estimation is considered, it is very important to understand the concept of theobservability of a system. Observability is used to determine whether the internal state ofa system can be inferred by knowledge about its external output. A formal definition ofobservability is the following [Kailtah, 1980].

Definition 5.1.1. Observability:A system is said to be observable if for any possible sequence of state and control vectors,the current state can be reconstructed in finite time as a function of only the past outputsequence.

Although Definition 5.1.1 is very clear, it has the drawback that is not straightfor-ward to check. Therefore, a more practical mathematical tool or condition is needed todetermine whether or not it is possible to estimate the state of a system.

Linear systems

Consider the SISO discrete linear system

xk+1 = Axk +Buk

yk = Cxk (5.1)

where xk ∈ Rn, uk ∈ R, yk ∈ R, and n is a positive scalar.Given the linear system (5.1), define the following observability matrix

O =

⎡⎢⎢⎢⎢⎢⎣

CCACA2

...CAn−1

⎤⎥⎥⎥⎥⎥⎦ . (5.2)

As shown in Kailtah [1980], or any other reference on linear system theory, using (5.2)the following definition is given.

Definition 5.1.2. Observable system:A time-invariant linear system in the state space representation (5.1) is observable if therank of the observability matrix (5.2) is equal to n, where n is the state dimension.

Definition 5.1.2 is straightforward to check, and this may be considered the first stepto do when state estimation has to be implemented for a linear system.

110





Linear systems


xk+1 = Axk +Buk

yk = Cxk (5.1)


O =

⎡⎢⎢⎢⎢⎢⎣

CCACA2

...CAn−1

⎤⎥⎥⎥⎥⎥⎦ . (5.2)




110





Linear systems


xk+1 = Axk +Buk

yk = Cxk (5.1)


O =

⎡⎢⎢⎢⎢⎢⎣

CCACA2

...CAn−1

⎤⎥⎥⎥⎥⎥⎦ . (5.2)




110





Linear systems


xk+1 = Axk +Buk

yk = Cxk (5.1)


O =

⎡⎢⎢⎢⎢⎢⎣

CCACA2

...CAn−1

⎤⎥⎥⎥⎥⎥⎦ . (5.2)




110

The state estimation problem

Nonlinear systems

The determination of observability for nonlinear systems is more complicated than theone for linear systems. The principal reason is that, for nonlinear systems, the observ-ability depends on the system input itself. In Hermann & Krener [1977] the property ofobservability for nonlinear systems is discussed extensively. While for linear systems theobservability is a unique and global property, for nonlinear systems this is not. In fact,there are four different forms of observability.

Given the general continuous time nonlinear system

x(t) = f(x(t),u(t)) (5.3a)y(t) = h(x(t)) (5.3b)

where x ∈ Rn, u ∈ Rm, and y ∈ Rp, the system may be

Locally observable ⇒ Observable⇓ ⇓

Locally weakly observable ⇒ Weakly observable

where the arrows show their relationships.Rather than stating all four observability definitions (they can be easily found in

Hermann & Krener [1977]) in this thesis the focus is only on the ‘locally weakly ob-servability’ property. This is mainly due to the fact that the locally weakly observabilityattribute is the only one which is possible to test in practice.

Thus, for the system (5.3), let us define an open set U contained in Rn, and thendefine the concept of ‘U -Indistinguishability’.

Definition 5.1.3 (Hermann & Krener [1977]). U -Indistinguishability:A pair of points x0 and x1, contained in U are called U -Indistinguishable if, for i = 0, 1,we have that solutions xi(t) of (5.3a) with respect to initial conditions xi(0) = xi areidentical, for every admissible control u(t) defined in the interval [0, T ].

Note that we denote all points x1 ∈ U that are U -Indistinguishable from x0 byI(x0,U).

Definition 5.1.4 (Hermann & Krener [1977]). The system (5.3) is locally weakly ob-servable at x0 if there exists an open neighborhood U of x0, such that for every openneighborhood V of x0 contained in U

I(x0,V) = x0 (5.4)

111


Nonlinear systems



x(t) = f(x(t),u(t)) (5.3a)y(t) = h(x(t)) (5.3b)










I(x0,V) = x0 (5.4)

111


Nonlinear systems



x(t) = f(x(t),u(t)) (5.3a)y(t) = h(x(t)) (5.3b)










I(x0,V) = x0 (5.4)

111


Nonlinear systems



x(t) = f(x(t),u(t)) (5.3a)y(t) = h(x(t)) (5.3b)










I(x0,V) = x0 (5.4)

111


Moreover

Definition 5.1.5 (Hermann & Krener [1977]). If U coincides with Rn, and Definition5.1.4 is valid for every x ∈ Rn, we say that the system (5.3) is locally weakly observ-able.

Locally weakly observability can be easily tested using a rank condition [Hermann& Krener, 1977; Besançon, 1999] similar to the one for linear systems.

For system (5.3) the following observability matrix

O(x) =

⎡⎢⎢⎢⎢⎢⎣

dh(x)d(Lfh(x))d(L2

fh(x))...

d(Ln−1f h(x))

⎤⎥⎥⎥⎥⎥⎦ (5.5)

is defined, where the differential of h is defined as

dh =∂h

∂x=

[∂h

∂x1

, . . . ,∂h

∂x1

], (5.6)

and for a constant input u

Lfh(x) =∂h

∂xf(x, u) (5.7)

defines the Lie derivative [Khalil, 2002] with the following property

Lifh(x) = LfL

i−1f h(x) =

∂(Li−1f )

∂xf(x, u) (5.8)

for i = 1, . . . , n− 1.Note that (5.5) is state dependent and has dimension n × n. The following are im-

portant theorems for testing the locally weakly observability.

Theorem 5.1.1 (Hermann & Krener [1977]). If the system (5.3) satisfies the rank condi-tion on (5.5) at x0 then (5.3) is locally weakly observable at x0.

In practice, the rank condition states that if, for a given x0, (5.5) has full rank n thenthe system (5.3) is locally weakly observable at x0. Note that the converse is almost al-ways true due to the following theorem.

Theorem 5.1.2 (Hermann & Krener [1977]). If the system (5.3) is weakly controllable,then it is weakly observable if and only if it is locally weakly observable if and only if theobservability rank condition is satisfied.

112


Moreover




O(x) =

⎡⎢⎢⎢⎢⎢⎣

dh(x)d(Lfh(x))d(L2

fh(x))...

d(Ln−1f h(x))

⎤⎥⎥⎥⎥⎥⎦ (5.5)


dh =∂h

∂x=

[∂h

∂x1

, . . . ,∂h

∂x1

], (5.6)


Lfh(x) =∂h

∂xf(x, u) (5.7)


Lifh(x) = LfL

i−1f h(x) =

∂(Li−1f )

∂xf(x, u) (5.8)






112


Moreover




O(x) =

⎡⎢⎢⎢⎢⎢⎣

dh(x)d(Lfh(x))d(L2

fh(x))...

d(Ln−1f h(x))

⎤⎥⎥⎥⎥⎥⎦ (5.5)


dh =∂h

∂x=

[∂h

∂x1

, . . . ,∂h

∂x1

], (5.6)


Lfh(x) =∂h

∂xf(x, u) (5.7)


Lifh(x) = LfL

i−1f h(x) =

∂(Li−1f )

∂xf(x, u) (5.8)






112


Moreover




O(x) =

⎡⎢⎢⎢⎢⎢⎣

dh(x)d(Lfh(x))d(L2

fh(x))...

d(Ln−1f h(x))

⎤⎥⎥⎥⎥⎥⎦ (5.5)


dh =∂h

∂x=

[∂h

∂x1

, . . . ,∂h

∂x1

], (5.6)


Lfh(x) =∂h

∂xf(x, u) (5.7)


Lifh(x) = LfL

i−1f h(x) =

∂(Li−1f )

∂xf(x, u) (5.8)






112

A solution: Kalman filtering

5.2 A solution: Kalman filteringIt is well known that the Kalman Filter (KF) [Kalman, 1960] is the optimal state estimatorfor unconstrained, linear systems subject to a normally distributed process and measure-ment noise. In this part of the thesis, general filter formulations are presented, more detailsand formulations are available in the literature. As a starting point several references aregiven during this discussion.

Given the following system

xk+1 = Akxk +Bkuk +Gkwk (5.9)yk = Ckxk + vk (5.10)

where xk ∈ Rn, uk ∈ Rm, yk ∈ Rp, and n, m, p are positive scalars, wk and vk aremutually independent sequences of zero mean white Gaussian noise with joint covariance

E

[wkw

Tk wkv

Tk

vkwTk vkv

Tk

]=

[P w

k 00 P v

k

]. (5.11)

Given, also, the initial conditions

x0 = E [x0] (5.12a)

P x0 = E

[(x0 − x0)(x0 − x0)

T]

(5.12b)

the Kalman Filter is obtained by minimizing the mean-square error between the state andits estimate. Generally, the KF algorithm is formulated in two steps, known as predictionand filtering. When the prediction step is running the filter computes the a priori stateestimate and its covariance

xk+1|k = Akxk|k +Bkuk, (5.13)

P xk+1|k = AkP

xk|kA

Tk +GkP

wk GT

k . (5.14)

When the new measurement (5.10) is available, the a priori statistics, (5.13) and (5.14),are updated by the filtering step

Kk = P xk|k−1C

Tk

(CkP

xk|k−1C

Tk + P v

k

)−1, (5.15)

xk|k = xk|k−1 +Kk

(yk −Ckxk|k−1

), (5.16)

P xk|k = [I −KkCk]P

xk|k−1, (5.17)

where Kk is the Kalman gain.Kalman Filter and its several extensions are well established, and successfully ap-

plied to solve estimation problems. Among them the Extended Kalman Filter and theUnscented Kalman Filter will be presented for the state estimation of nonlinear systems.

For a more detailed Kalman Filter presentation, including the case where the cross-covariance terms vkw

Tk and wkv

Tk are not zero, the reader is referred to Simon [2006].

113






E

[wkw

Tk wkv

Tk

vkwTk vkv

Tk

]=

[P w

k 00 P v

k

]. (5.11)


x0 = E [x0] (5.12a)

P x0 = E

[(x0 − x0)(x0 − x0)

T]

(5.12b)



P xk+1|k = AkP

xk|kA

Tk +GkP

wk GT

k . (5.14)


Kk = P xk|k−1C

Tk

(CkP

xk|k−1C

Tk + P v

k

)−1, (5.15)

xk|k = xk|k−1 +Kk

(yk −Ckxk|k−1

), (5.16)


xk|k−1, (5.17)




Tk and wkv


113






E

[wkw

Tk wkv

Tk

vkwTk vkv

Tk

]=

[P w

k 00 P v

k

]. (5.11)


x0 = E [x0] (5.12a)

P x0 = E

[(x0 − x0)(x0 − x0)

T]

(5.12b)



P xk+1|k = AkP

xk|kA

Tk +GkP

wk GT

k . (5.14)


Kk = P xk|k−1C

Tk

(CkP

xk|k−1C

Tk + P v

k

)−1, (5.15)

xk|k = xk|k−1 +Kk

(yk −Ckxk|k−1

), (5.16)


xk|k−1, (5.17)




Tk and wkv


113






E

[wkw

Tk wkv

Tk

vkwTk vkv

Tk

]=

[P w

k 00 P v

k

]. (5.11)


x0 = E [x0] (5.12a)

P x0 = E

[(x0 − x0)(x0 − x0)

T]

(5.12b)



P xk+1|k = AkP

xk|kA

Tk +GkP

wk GT

k . (5.14)


Kk = P xk|k−1C

Tk

(CkP

xk|k−1C

Tk + P v

k

)−1, (5.15)

xk|k = xk|k−1 +Kk

(yk −Ckxk|k−1

), (5.16)


xk|k−1, (5.17)




Tk and wkv


113


5.2.1 Extended Kalman FilterThe Extended Kalman Filter (EKF) is perhaps the most famous extension of the KF fornonlinear systems. There are several versions available in the literature, see for exampleSimon [2006].

Consider the following nonlinear system

xk+1 = f(xk,uk,wk) (5.18a)yk = h(xk,vk) (5.18b)

where xk ∈ Rn, uk ∈ Rm, yk ∈ Rp, n, m, p are positive scalars, wk and vk are normallydistributed Gaussian noise sequences with joint covariance (5.11).

The EKF gives an approximation of the optimal state estimate, and the nonlinearitiesof the system (5.18) are approximated by a linearized version of the nonlinear modelaround the last state estimate. For this approximation to be valid, it is very importantthat the linearization is a good representation of the nonlinear model in all the uncertaintydomain associated with the state estimate.

Given initial conditions

x0 = E[x0] (5.19a)

P x0 = E[(x0 − x0)(x0 − x0)

T ] (5.19b)

for each sample k = 1, ...,∞, the EKF algorithm is formulated by the following steps.

• Calculate the following Jacobians around the last filtered state estimate xk|k

Ak =∂f(x,u,w)

∂x

∣∣∣∣x=xk|k

Gk =∂f(x,u,w)

∂w

∣∣∣∣w=w

(5.20)

where w is the mean value of the process noise sequence wk.• Apply the prediction step of the Kalman Filter

xk+1|k = f(xk|k,uk, w) (5.21)

P xk+1|k = AkP

xk|kA

Tk +GkP

wk GT

k (5.22)

• Calculate the following Jacobians around the predicted state estimate xk+1|k

Ck =∂h(x,v)

∂x

∣∣∣∣x=xk+1|k

Lk =∂h(x,v)

∂v

∣∣∣∣v=v

(5.23)

where v is the mean value of the measurement noise sequence vk.

114








x0 = E[x0] (5.19a)

P x0 = E[(x0 − x0)(x0 − x0)

T ] (5.19b)



Ak =∂f(x,u,w)

∂x

∣∣∣∣x=xk|k

Gk =∂f(x,u,w)

∂w

∣∣∣∣w=w

(5.20)


xk+1|k = f(xk|k,uk, w) (5.21)

P xk+1|k = AkP

xk|kA

Tk +GkP

wk GT

k (5.22)


Ck =∂h(x,v)

∂x

∣∣∣∣x=xk+1|k

Lk =∂h(x,v)

∂v

∣∣∣∣v=v

(5.23)


114








x0 = E[x0] (5.19a)

P x0 = E[(x0 − x0)(x0 − x0)

T ] (5.19b)



Ak =∂f(x,u,w)

∂x

∣∣∣∣x=xk|k

Gk =∂f(x,u,w)

∂w

∣∣∣∣w=w

(5.20)


xk+1|k = f(xk|k,uk, w) (5.21)

P xk+1|k = AkP

xk|kA

Tk +GkP

wk GT

k (5.22)


Ck =∂h(x,v)

∂x

∣∣∣∣x=xk+1|k

Lk =∂h(x,v)

∂v

∣∣∣∣v=v

(5.23)


114








x0 = E[x0] (5.19a)

P x0 = E[(x0 − x0)(x0 − x0)

T ] (5.19b)



Ak =∂f(x,u,w)

∂x

∣∣∣∣x=xk|k

Gk =∂f(x,u,w)

∂w

∣∣∣∣w=w

(5.20)


xk+1|k = f(xk|k,uk, w) (5.21)

P xk+1|k = AkP

xk|kA

Tk +GkP

wk GT

k (5.22)


Ck =∂h(x,v)

∂x

∣∣∣∣x=xk+1|k

Lk =∂h(x,v)

∂v

∣∣∣∣v=v

(5.23)


114


• Apply the filtering step to the linearized observation dynamics

Kk = P xk|k−1C

Tk

(CkP

xk|k−1C

Tk +LkP

vkL

Tk

)−1, (5.24)

xk|k = xk|k−1 +Kk

(yk − h(xk|k−1, v)

), (5.25)


xk|k−1, (5.26)

where Kk is the Kalman gain.

The EKF is not an optimal filter, but rather it is implemented based on approximations.This implies that the matrix P x is not the true covariance but an approximation. While theKalman Filter for linear systems is an optimal state estimator, which always provides stateestimate convergence, if the observability condition (5.1.2) is met, the EKF may divergeif the approximation due the the consecutive linearization is not sufficiently accurate.

5.2.2 Unscented Kalman FilterA more recent approach than the EKF, to the state estimation problem, is the UnscentedKalman Filter. The UKF uses the Unscented Transformation, which is based on the ideathat is easier to approximate a probability distribution than an arbitrary nonlinear functionor transformation [Julier & Uhlmann, 1996]. This approximation is done using a finiteset of points, called sigma points. An important feature of the UKF, with respect to theEKF, is that no Jacobians need to be computed. This is relevant especially in the case ofstrong nonlinearities, as the introduction of linearization errors is avoided. In general bothfilters have similar computational complexity Wan & Van Der Merwe [2000], and bothimplementations are straightforward.

In the following a UKF formulation is presented, more precisely the one used in Wan& Van Der Merwe [2001].

Given the nonlinear system (5.18), define L = n+ dim(w)+ dim(v), where dim(·)indicates the dimension of (·). Define the following scalar weights Wi

W(m)0 =

λ

(L+ λ)(5.27a)

W(c)0 =

λ

(L+ λ)+ (1− α2 + β) (5.27b)

W(m)i = W

(c)i =

1

2(L+ λ), i = 1, ...2L (5.27c)

with λ = α2(L + κ) − L. Note that these weights may be positive, or negative, but toprovide an unbiased estimate,

∑2L+1i W

(m)i = 1 must be satisfied. The design parameters

α and κ control the spread of sigma points, β is related to the distribution of the random

115



Kk = P xk|k−1C

Tk

(CkP

xk|k−1C

Tk +LkP

vkL

Tk

)−1, (5.24)

xk|k = xk|k−1 +Kk

(yk − h(xk|k−1, v)

), (5.25)


xk|k−1, (5.26)






W(m)0 =

λ

(L+ λ)(5.27a)

W(c)0 =

λ

(L+ λ)+ (1− α2 + β) (5.27b)

W(m)i = W

(c)i =

1

2(L+ λ), i = 1, ...2L (5.27c)


∑2L+1i W



115



Kk = P xk|k−1C

Tk

(CkP

xk|k−1C

Tk +LkP

vkL

Tk

)−1, (5.24)

xk|k = xk|k−1 +Kk

(yk − h(xk|k−1, v)

), (5.25)


xk|k−1, (5.26)






W(m)0 =

λ

(L+ λ)(5.27a)

W(c)0 =

λ

(L+ λ)+ (1− α2 + β) (5.27b)

W(m)i = W

(c)i =

1

2(L+ λ), i = 1, ...2L (5.27c)


∑2L+1i W



115



Kk = P xk|k−1C

Tk

(CkP

xk|k−1C

Tk +LkP

vkL

Tk

)−1, (5.24)

xk|k = xk|k−1 +Kk

(yk − h(xk|k−1, v)

), (5.25)


xk|k−1, (5.26)






W(m)0 =

λ

(L+ λ)(5.27a)

W(c)0 =

λ

(L+ λ)+ (1− α2 + β) (5.27b)

W(m)i = W

(c)i =

1

2(L+ λ), i = 1, ...2L (5.27c)


∑2L+1i W



115


variable x. In most cases typical values are β = 2, and κ = 0 or κ = 3− n, leaving onlythe parameter α as design parameter. Considering that 1 · 10−5 ≤ α ≤ 1, the tuning ofthe UKF becomes simpler. For finer tuning and a more comprehensive description of theUKF parameters see Wan & Van Der Merwe [2001]. As it is the case for the EKF, theinitial covariance matrices can be also used for performance tuning.


x0 = E[x0] (5.28a)

P x0 = E[(x0 − x0)(x0 − x0)

T ], (5.28b)

the modified nonlinear dynamics matrix F(X x

j ,uj,Xwj

)[f(X x(0)

j ,uj,Xw(0)j

)f(X x(1)

j ,uj,Xw(1)j

). . . f

(X x(2L+1)

j ,uj,Xw(2L+1)j

),]

(5.29)

and the observation mapping H(X x

j ,X vj

)[h(X x(0)

j ,uj,Xw(0)j

)h(X x(1)

j ,uj,Xw(1)j

). . . h

(X x(2L+1)

j ,uj,Xw(2L+1)j

),]

(5.30)

where f(·) and h(·) are defined in (5.18b); the subscript j indicates the sampling timeindex; the superscript (i) for i = (0, 1, . . . , 2L + 1) and the sigma points X j are definedin (5.32-5.33).

For each sample k = 1, ...,∞, the UKF algorithm is formulated as follow.

• Augment the state vector and the covariance matrix

xak = E[xa

k] = [xTk 0T

w 0Tv ]

T (5.31a)

P ak =

⎡⎣P x

k 0 00 P w

k 00 0 P v

k

⎤⎦ (5.31b)

• Calculate the sigma point matrix X ak−1, where its i-th column vector X a(i)

k−1 is de-fined as

X a(0)k−1 = xk−1 (5.32a)

X a(i)k−1 = xk−1 + γ

(√P a

k−1

)i

i = 1, . . . , L (5.32b)

X a(i)k−1 = xk−1 − γ

(√P a

k−1

)i−L

i = L+ 1, . . . , 2L (5.32c)

116




x0 = E[x0] (5.28a)

P x0 = E[(x0 − x0)(x0 − x0)

T ], (5.28b)


j ,uj,Xwj

)[f(X x(0)

j ,uj,Xw(0)j

)f(X x(1)

j ,uj,Xw(1)j

). . . f

(X x(2L+1)

j ,uj,Xw(2L+1)j

),]

(5.29)


j ,X vj

)[h(X x(0)

j ,uj,Xw(0)j

)h(X x(1)

j ,uj,Xw(1)j

). . . h

(X x(2L+1)

j ,uj,Xw(2L+1)j

),]

(5.30)




xak = E[xa

k] = [xTk 0T

w 0Tv ]

T (5.31a)

P ak =

⎡⎣P x

k 0 00 P w

k 00 0 P v

k

⎤⎦ (5.31b)



X a(0)k−1 = xk−1 (5.32a)

X a(i)k−1 = xk−1 + γ

(√P a

k−1

)i

i = 1, . . . , L (5.32b)

X a(i)k−1 = xk−1 − γ

(√P a

k−1

)i−L

i = L+ 1, . . . , 2L (5.32c)

116




x0 = E[x0] (5.28a)

P x0 = E[(x0 − x0)(x0 − x0)

T ], (5.28b)


j ,uj,Xwj

)[f(X x(0)

j ,uj,Xw(0)j

)f(X x(1)

j ,uj,Xw(1)j

). . . f

(X x(2L+1)

j ,uj,Xw(2L+1)j

),]

(5.29)


j ,X vj

)[h(X x(0)

j ,uj,Xw(0)j

)h(X x(1)

j ,uj,Xw(1)j

). . . h

(X x(2L+1)

j ,uj,Xw(2L+1)j

),]

(5.30)




xak = E[xa

k] = [xTk 0T

w 0Tv ]

T (5.31a)

P ak =

⎡⎣P x

k 0 00 P w

k 00 0 P v

k

⎤⎦ (5.31b)



X a(0)k−1 = xk−1 (5.32a)

X a(i)k−1 = xk−1 + γ

(√P a

k−1

)i

i = 1, . . . , L (5.32b)

X a(i)k−1 = xk−1 − γ

(√P a

k−1

)i−L

i = L+ 1, . . . , 2L (5.32c)

116




x0 = E[x0] (5.28a)

P x0 = E[(x0 − x0)(x0 − x0)

T ], (5.28b)


j ,uj,Xwj

)[f(X x(0)

j ,uj,Xw(0)j

)f(X x(1)

j ,uj,Xw(1)j

). . . f

(X x(2L+1)

j ,uj,Xw(2L+1)j

),]

(5.29)


j ,X vj

)[h(X x(0)

j ,uj,Xw(0)j

)h(X x(1)

j ,uj,Xw(1)j

). . . h

(X x(2L+1)

j ,uj,Xw(2L+1)j

),]

(5.30)




xak = E[xa

k] = [xTk 0T

w 0Tv ]

T (5.31a)

P ak =

⎡⎣P x

k 0 00 P w

k 00 0 P v

k

⎤⎦ (5.31b)



X a(0)k−1 = xk−1 (5.32a)

X a(i)k−1 = xk−1 + γ

(√P a

k−1

)i

i = 1, . . . , L (5.32b)

X a(i)k−1 = xk−1 − γ

(√P a

k−1

)i−L

i = L+ 1, . . . , 2L (5.32c)

116


where γ =√L+ λ, and

(√P a

k−1

)i

is the i-th column vector of the square root ofthe covariance matrix P a

k−1. Note that X ak−1 has dimension L × (2L + 1) and is

partitioned as

X ak−1 =

⎡⎣X x

k−1

Xwk−1

X vk−1

⎤⎦ (5.33)

with respect to row vectors.• Propagate the sigma points through the nonlinear dynamics matrix defined in (5.29),

properly modified to allow the correct sigma point propagation, and compute thepredicted state estimate, where the index i is used to select the appropriate sigmapoint column

X xk|k−1 = F

(X xk−1,uk−1,Xw

k−1

)(5.34)

x−k =

2L∑i=0

W(m)i X x(i)

k|k−1. (5.35)

Note that the a priori state estimate x−k is found as a weighted sum of the propagated

sigma points.• Compute the predicted covariance, instantiate the prediction points through the ob-

servation mapping defined in (5.30). This is properly modified to allow the correctsigma point propagation, compute the predicted state estimate, and calculate thepredicted measurement

P x−k =

2L∑i=0

W(c)i

[X x(i)

k|k−1 − x−k

] [X x(i)

k|k−1 − x−k

]T(5.36)

Yk|k−1 = H(X x

k|k−1,X vk|k−1

)(5.37)

y−k =

2L∑i=0

W(m)i Y (i)

k|k−1. (5.38)

• Obtain the innovation covariance and the cross covariance matrices

Pyk yk=

2L∑i=0

W(c)i

[Y (i)

k|k−1 − y−k

] [Y (i)

k|k−1 − y−k

]T(5.39)

Pykxk=

2L∑i=0

W(c)i

[X (i)

k|k−1 − x−k

] [Y (i)

k|k−1 − y−k

]T. (5.40)

117



(√P a

k−1

)i



partitioned as

X ak−1 =

⎡⎣X x

k−1

Xwk−1

X vk−1

⎤⎦ (5.33)



X xk|k−1 = F

(X xk−1,uk−1,Xw

k−1

)(5.34)

x−k =

2L∑i=0

W(m)i X x(i)

k|k−1. (5.35)




P x−k =

2L∑i=0

W(c)i

[X x(i)

k|k−1 − x−k

] [X x(i)

k|k−1 − x−k

]T(5.36)

Yk|k−1 = H(X x

k|k−1,X vk|k−1

)(5.37)

y−k =

2L∑i=0

W(m)i Y (i)

k|k−1. (5.38)


Pyk yk=

2L∑i=0

W(c)i

[Y (i)

k|k−1 − y−k

] [Y (i)

k|k−1 − y−k

]T(5.39)

Pykxk=

2L∑i=0

W(c)i

[X (i)

k|k−1 − x−k

] [Y (i)

k|k−1 − y−k

]T. (5.40)

117



(√P a

k−1

)i



partitioned as

X ak−1 =

⎡⎣X x

k−1

Xwk−1

X vk−1

⎤⎦ (5.33)



X xk|k−1 = F

(X xk−1,uk−1,Xw

k−1

)(5.34)

x−k =

2L∑i=0

W(m)i X x(i)

k|k−1. (5.35)




P x−k =

2L∑i=0

W(c)i

[X x(i)

k|k−1 − x−k

] [X x(i)

k|k−1 − x−k

]T(5.36)

Yk|k−1 = H(X x

k|k−1,X vk|k−1

)(5.37)

y−k =

2L∑i=0

W(m)i Y (i)

k|k−1. (5.38)


Pyk yk=

2L∑i=0

W(c)i

[Y (i)

k|k−1 − y−k

] [Y (i)

k|k−1 − y−k

]T(5.39)

Pykxk=

2L∑i=0

W(c)i

[X (i)

k|k−1 − x−k

] [Y (i)

k|k−1 − y−k

]T. (5.40)

117



(√P a

k−1

)i



partitioned as

X ak−1 =

⎡⎣X x

k−1

Xwk−1

X vk−1

⎤⎦ (5.33)



X xk|k−1 = F

(X xk−1,uk−1,Xw

k−1

)(5.34)

x−k =

2L∑i=0

W(m)i X x(i)

k|k−1. (5.35)




P x−k =

2L∑i=0

W(c)i

[X x(i)

k|k−1 − x−k

] [X x(i)

k|k−1 − x−k

]T(5.36)

Yk|k−1 = H(X x

k|k−1,X vk|k−1

)(5.37)

y−k =

2L∑i=0

W(m)i Y (i)

k|k−1. (5.38)


Pyk yk=

2L∑i=0

W(c)i

[Y (i)

k|k−1 − y−k

] [Y (i)

k|k−1 − y−k

]T(5.39)

Pykxk=

2L∑i=0

W(c)i

[X (i)

k|k−1 − x−k

] [Y (i)

k|k−1 − y−k

]T. (5.40)

117


• Perform the measurement update

Kk = PykxkP−1

yk yk, (5.41)

xk = x−k +Kk

(yk − y−

k

), (5.42)

P xk = P x−

k −KkPyk ykKT

k . (5.43)

Note that no linearization procedure is required. Moreover, a square root of ma-trix,

√P a

k−1, has to be calculated at every time step, thus a numerical stable and efficientalgorithm must be used, for example the Cholesky factorization. This is the most compu-tationally demanding operation. A more efficient implementation is the square-root UKF[Van Der Merwe & Wan, 2001] which uses a recursive form of the Cholesky factorization.

The algorithm implementation is straightforward, since only simple operations needto be performed, e.g. weighted sums. Other UKF formulations are available in the lit-erature, see for instance Wan & Van Der Merwe [2001] where an algorithm that uses areduced number of sigma points is shown. For example, this is well suited when processand measurement noise realizations are assumed to be additive.

In general, UKF gives an accuracy of the second-order for the state estimate. Thisis the same as the standard EKF algorithm approach. However, for Gaussian distribution,a particular choice of the UKF parameter β, i.e. β = 2, gives an accuracy of the fourth-order term in the Taylor series expansion of the covariance, as discussed in the appendixof Julier & Uhlmann [2004].

Some successful applications of the UKF are described in Kandepu et al. [2008],where several comparisons with the EKF are discussed, for standard examples used inthe literature, such as Van der Pol oscillator, and also for a solid oxide fuel cell combinedgas turbine hybrid system. Van Der Merwe et al. [2004] give an interesting discussionof a family of Sigma-point based filters and compares their performance with the EKF,for an unmanned aerial vehicle where GPS measurements are integrated with a inertialmeasurement unit. An interesting discussion on UKF estimation and noise modeling isincluded in Kolås [2008]. Finally, in Spivey et al. [2010] the UKF is applied to an indus-trial process fouling and its performance is compared with Moving Horizon Estimationand EKF for a set of experimental data.

In next sections, the advantages of UKF compared to EKF are illustrated in twoexamples. Firstly, estimation for a microalgae photobioreactor is given. The results pre-sented are particularly interesting since the performance of both estimators are comparedwith respect to real data obtained from laboratory experiments. Secondly, a well con-structed example shows how both filters perform when a locally weakly unobservablenonlinear system is considered. This is shown in an NMPC framework.

118



Kk = PykxkP−1

yk yk, (5.41)

xk = x−k +Kk

(yk − y−

k

), (5.42)

P xk = P x−

k −KkPyk ykKT

k . (5.43)


√P a






118



Kk = PykxkP−1

yk yk, (5.41)

xk = x−k +Kk

(yk − y−

k

), (5.42)

P xk = P x−

k −KkPyk ykKT

k . (5.43)


√P a






118



Kk = PykxkP−1

yk yk, (5.41)

xk = x−k +Kk

(yk − y−

k

), (5.42)

P xk = P x−

k −KkPyk ykKT

k . (5.43)


√P a






118

Unscented and Extended Kalman filtering for a photobioreactor

5.3 Unscented and Extended Kalman filtering for a pho-tobioreactor

Frequently, in the chemical and biochemical process industry, there is the necessity tomonitor and control chemical reactions. This is usually done by using sensors that givemeasurements. Typically, measurements of reactant and product concentrations, operat-ing temperatures, pressures, and other parameters are obtained. In general, a measurementhas to be reliable, i.e. it has to be available and accurate. However, there are several rea-sons why the required measurements may not be reliable. Some of these reasons are theimpracticability of building an appropriate sensor due to lack of technology, the diffi-culty of the positioning the sensor, the associated cost. In such cases, an attempt to useestimation techniques may be necessary.

Dochain [2003] presents an interesting overview of available results on state andparameter estimation in chemical and biochemical processes. A comparison of severaltraditional state and parameter estimation approaches is given, discussing the pros andcons of different cases, and describing how the most common implementation problemsare solved (see Dochain [2003] and references therein).

In this section, an application of the UKF to a photobioreactor for microalgae produc-tion is shown. Microalgae have many applications such as the production of high valuecompounds (source of long-chain polyunsaturated fatty acids, vitamins, and pigments),in energy production (e.g. microalgae hydrogen, biofuel, methane) or in environmentalremediation (especially carbon dioxide fixation and greenhouse gas emissions reduction).However, the photobioreactor microalgae process needs complex and costly hardware,especially for biomass measurement. There are also problems with the practicabilityof finding reliable online sensors that are able to measure the state variables (Shimizu[1996]). Thus, state and parameter estimation seems to be a critical issue and is studiedin the case of a culture of the microalgae Porphyridium purpureum.

The intention of this work is to present the advantages of UKF in terms of perfor-mance and implementation ease, compared to the EKF. The work of Becerra-Celis et al.[2008] is considered as starting point. Therein it is shown how to implement an EKF forstate estimation in a photobioreactor. Due to the operational data availability for biomassmeasurement, the results are thus validated. Numerical simulations in batch mode, andreal-life experiments in continuous mode are given in order to highlight the performanceof the proposed estimator. For the state estimation work on the photobioreactor presentedin this chapter, both the model and the experimental data used for validating the result aretaken from Becerra-Celis et al. [2008]

119







119







119







119


5.3.1 Photobioreactor for microalgae production

Strain and growth conditions

The photobioreactor is used to produce the red microalgae Porphyridium purpureum SAG1830-1A obtained from the Sammlung von Algenkulture Pflanzenphysiologister Insti-tut Universität Göttingen, Germany. The strain is grown and maintained on Hemerickmedium (Hemerick [1973]). The pH of the Hemerick medium is adjusted to 7.0 beforeautoclaving it for 20 minutes at 121 ◦C. Cultures are maintained at 25 ◦C in 500 ml flaskcontaining 400 ml culture under continuous light intensity of 70 μEm−2s−1 and aeratedwith air containing 1% (v/v) CO2 at 100 rpm on an orbital shaker. During the exponen-tial growth phase, within an interval of two weeks, 200 ml of culture are transferred to anew flask containing fresh medium.

Culture conditions and measurements

Figure 5.1 illustrates the photobioreactor diagram where the growth of cultures is per-formed. The bubble column photobioreactor has a working height of 0.4 m and a diame-ter of 0.1 m. The total culture volume is 2.5 l, and the cylindrical reactor, made of glass,has an illuminated area of 0.1096 m2. To agitate the culture an air mixture with 2% (v/v)CO2 is continuously supplied at a flow rate of 2.5 V.V.H (gas volume per liquid culturevolume per hour). 0.22 μm Millipore filters, appropriate valves and flowmeters are usedto filter and to control the air flow rate entering the photobioreactor. Four OSRAM whitefluorescent tubes (L30W/72) and three OSRAM pink fluorescent tubes (L30W/77) arearranged around the bubble column to provide the external light source. The incidentlight intensity on the reactor surface is measured at ten different locations with flat sur-face quantum sensors (LI-COR LI-190SA). The average light intensity is computed bythe weighted average of all measurements. The optimal value of irradiance on surfacefor the reactor is found to be 120 μEm−2s−1. A transparent jacket connected to a ther-mostat unit enables the temperature control, regulated to 25 ◦C. Other sensors are a pHsensor (Radiometer Analytical) and a dissolved oxygen sensor (Ingold type 170). A sam-pling port is applied to the top of the column, from where samples for off line analysisare collected after 6, 8, and 12 hours. The number of cells is counted using an opticalmicroscope ZEISS Axioplan-2 on Malassez cells. The total inorganic carbon (T.I.C.) inthe culture medium is calculated by gas phase chromatography. This method, proposedby Marty et al. [1995], is used to measure low inorganic carbon concentrations down to(10−6mol l−1) within an accuracy of 10 %.

120







120







120







120


��

��

��

��

��

� ��

� ��

��

��

��

��

��

��

��

� !

��

��

��

��

"��# �

�

Figure 5.1: Photobioreactor diagram.

121


��

��

��

��

��

� ��

� ��

��

��

��

��

��

��

��

� !

��

��

��

��

"��# �

�


121


��

��

��

��

��

� ��

� ��

��

��

��

��

��

��

��

� !

��

��

��

��

"��# �

�


121


��

��

��

��

��

� ��

� ��

��

��

��

��

��

��

��

� !

��

��

��

��

"��# �

�


121


Mathematical model

The bioprocess model presented in Baquerisse et al. [1999] is used. It consists of twosub models, one describing the growth kinetics, and the other representing the gas-liquidmass transfer in the photobioreactor. This results in two differential equations describingthe state of the reactor

dX

dt=

Fin

VXin + μX − Fout

VX (5.44a)

d[TIC]

dt=

Fin

V[TIC]in − Fout

V[TIC]out − μ

X

YX/S

mX + kLa ([CO∗2]− [CO2])

(5.44b)

where X is the biomass, and [TIC] is the inorganic carbon concentration. The subscripts[·]in and [·]out indicate quantities flowing into, and out from the reactor, respectively. V isthe culture volume, and F is the medium flow rate. The mass conversion yield is definedby YX/S , m is the maintenance coefficient, and kLa is the gas-liquid transfer coefficient.The carbon dioxide concentration for the fresh medium is defined as

[CO∗2] =

PCO2

H (5.45)

where PCO2 is the partial pressure of carbon dioxide, and H is the Henry’s constant forHemerick medium. Moreover, the carbon dioxide concentration in the medium is givenby:

[CO2] =[TIC][

1 + K1

[H+]+ K1K2

[H+]2

] (5.46)

where K1, K2 are dissociation equilibrium constants, and [H+] is defined as

[H+] = 10−pH (5.47)

representing the hydrogen ions concentration in the culture media.In addition, a light transfer model is considered, which describes the evolution of

incident and outgoing light intensity

E =(Iin − Iout)Ar

V X, (5.48)

Iout = C1IinXC2 , (5.49)

122


Mathematical model


dX

dt=

Fin

VXin + μX − Fout

VX (5.44a)

d[TIC]

dt=

Fin

V[TIC]in − Fout

V[TIC]out − μ

X

YX/S

mX + kLa ([CO∗2]− [CO2])

(5.44b)


[CO∗2] =

PCO2

H (5.45)


[CO2] =[TIC][

1 + K1

[H+]+ K1K2

[H+]2

] (5.46)


[H+] = 10−pH (5.47)



E =(Iin − Iout)Ar

V X, (5.48)


122


Mathematical model


dX

dt=

Fin

VXin + μX − Fout

VX (5.44a)

d[TIC]

dt=

Fin

V[TIC]in − Fout

V[TIC]out − μ

X

YX/S

mX + kLa ([CO∗2]− [CO2])

(5.44b)


[CO∗2] =

PCO2

H (5.45)


[CO2] =[TIC][

1 + K1

[H+]+ K1K2

[H+]2

] (5.46)


[H+] = 10−pH (5.47)



E =(Iin − Iout)Ar

V X, (5.48)


122


Mathematical model


dX

dt=

Fin

VXin + μX − Fout

VX (5.44a)

d[TIC]

dt=

Fin

V[TIC]in − Fout

V[TIC]out − μ

X

YX/S

mX + kLa ([CO∗2]− [CO2])

(5.44b)


[CO∗2] =

PCO2

H (5.45)


[CO2] =[TIC][

1 + K1

[H+]+ K1K2

[H+]2

] (5.46)


[H+] = 10−pH (5.47)



E =(Iin − Iout)Ar

V X, (5.48)


122


where E is the light "energy" accessible per cell, Iout is the outgoing light intensity, Iin isthe ingoing light intensity. C1, C2 are constants depending on the reactor geometry, andAr is its area.

The light intensity and the total carbon concentration influence the specific growthrate

μ = μmaxE

Eopt

e

(1− E

Eopt

)[TIC]

[TIC]opte

(1− [TIC]

[TIC]opt

)(5.50)

where μmax, Eopt, and [TIC]opt are model parameters identified from the batch data ex-periments. Finally, the substrate limitation effect, which is the tendency of cells to dis-tribute in layers, is taken into account in (5.50). An example of substrate limitation iswhen cells accumulate close to the light source.

Batch and continuous operating conditions

The photobioreactor can work in two different operating conditions, batch mode and con-tinuous mode. In batch mode:

Fin = Fout = 0; [TIC]in = 0; Xin = 0. (5.51)

In continuous mode, instead:

Fin = Fout �= 0. (5.52)

Model parameters

The model parameters used in this work are the ones identified in Becerra-Celis et al.[2008]. For more details on the system identification procedure the reader is referred totheir work. Tables 5.1 and 5.2 contain the parameters for the microalgae and the totalinorganic carbon dynamics, respectively.

Table 5.1: Model parameters for Porphyridium purpureum at 25 ◦C.Parameter Unit Value

μmax h−1 0.0337Eopt μEs−1(109cell)−1 1.20

[TIC]opt mmolel−1 12.93C1 0.28C2 −0.55

123




μ = μmaxE

Eopt

e

(1− E

Eopt

)[TIC]

[TIC]opte

(1− [TIC]

[TIC]opt

)(5.50)






Fin = Fout �= 0. (5.52)

Model parameters




[TIC]opt mmolel−1 12.93C1 0.28C2 −0.55

123




μ = μmaxE

Eopt

e

(1− E

Eopt

)[TIC]

[TIC]opte

(1− [TIC]

[TIC]opt

)(5.50)






Fin = Fout �= 0. (5.52)

Model parameters




[TIC]opt mmolel−1 12.93C1 0.28C2 −0.55

123




μ = μmaxE

Eopt

e

(1− E

Eopt

)[TIC]

[TIC]opte

(1− [TIC]

[TIC]opt

)(5.50)






Fin = Fout �= 0. (5.52)

Model parameters




[TIC]opt mmolel−1 12.93C1 0.28C2 −0.55

123


Table 5.2: Model parameters for [TIC] dynamics.Parameter Unit Value

K1 1.02 · 10−6

K2 8.32 · 10−10

kLa h−1 41.40m h−1mmole(109cell)−1 0.004

YX/S 109 cell per mole TIC 198.1H atm l mole−1 34.03

5.3.2 Biomass estimation

Using an appropriate numerical integration routine, the photobioreactor model (5.44) canbe written in the following form

ξk+1 = f(ξk, uk,wξk;ηk) (5.53a)

ηk+1 = ηk +wηk (5.53b)

yk = [0 1]ξk + vk (5.53c)

where the state vector is

ξk =

[Xk

[TIC]k

](5.54)

uk = Fin is the input, yk = [TIC] is the measurement, vξk is the process noise, vk

is the measurement noise, of appropriate dimensions, respectively. Here, (5.53b) is theparameter equation, where the parameter dynamics is modeled as a random walk driven bya white noise process wη

k . A relatively small covariance is associated to wηk to consider the

slowly varying nature of the parameter vector. Finally, when the measurement of PCO2,pH , and Iin are available, they are used for updating model parameters.

It may happen that parameters used in the model (5.53) are uncertain or inaccurate.This would decrease the estimator model accuracy. One method to obtain sufficientlygood estimate is to make the estimator algorithm robust with respect to the parametervariations. Another method is to try estimating the uncertain parameters for updating themodel, and thereby obtaining better accuracy. However, joint parameter and state estima-tion may lead to observability problems. Therefore one has to be careful about choosinga subset of parameter to estimate, so that the system observability is not penalized. Ageneral framework to introduce the parameter estimation is to extend the state with theuncertain parameters vector and then estimate the augmented state, as shown next for theUKF case.

124



K1 1.02 · 10−6

K2 8.32 · 10−10






ηk+1 = ηk +wηk (5.53b)

yk = [0 1]ξk + vk (5.53c)


ξk =

[Xk

[TIC]k

](5.54)






124



K1 1.02 · 10−6

K2 8.32 · 10−10






ηk+1 = ηk +wηk (5.53b)

yk = [0 1]ξk + vk (5.53c)


ξk =

[Xk

[TIC]k

](5.54)






124



K1 1.02 · 10−6

K2 8.32 · 10−10






ηk+1 = ηk +wηk (5.53b)

yk = [0 1]ξk + vk (5.53c)


ξk =

[Xk

[TIC]k

](5.54)






124


Modified Unscented Kalman Filter algorithm

To use the UKF for joint state and parameter estimation, the algorithm (5.28-5.43) mustbe modified as follow.

Define the new state vector

xk =

[ξkηk

](5.55)

which has as elements, the state and parameter estimates, respectively. The vector

wk =

[wξ

k

wηk

](5.56)

contains the process noises in the evolution of ξ and η. Analogously, define the aug-mented covariance matrix

Pk =

[P x

k P x,wk

P w,xk P w

k

](5.57)

where P xk consists of the state and parameter error covariances, while P w

k includes theprocess noise covariance associated to state and parameters. In addition, the off diag-onal entries are cross covariance terms represented by the notation P ·,·

k . Obviously, allelements of xk and Pk are of appropriate dimensions.

Given the initial conditions

xk =

[ξkηk

]P0 =

[P x

0 00 P w

0

](5.58)

where the new system state x has dimension n, it is possible to apply the UKF algorithm(5.28-5.43) and jointly estimate the state and parameter vectors.

Simulation results with experimental data

Due to modifications just introduced, the UKF described in Section 5.3.2 can be imple-mented. The main objective is to estimate the biomass X in the photobioreactor of Section5.3.1. Focusing on the two different working conditions defined in Section 5.3.1, it is ob-served how in batch mode the UKF has excellent performance, which also is the case forthe EKF designed in Becerra-Celis et al. [2008]. This is due to the fact that the modelparameters are identified in batch mode, and the measurements have a constant samplingtime. A more complex scenario appears for continuous cultures. The model parametersare still the ones from the batch experiments, and the experimental data are collected atvariable instant intervals. Due to the variable time steps, Becerra-Celis et al. [2008] im-plement a continuous discrete version of the EKF. In the present work, this problem is

125





xk =

[ξkηk

](5.55)


wk =

[wξ

k

wηk

](5.56)


Pk =

[P x

k P x,wk

P w,xk P w

k

](5.57)





xk =

[ξkηk

]P0 =

[P x

0 00 P w

0

](5.58)




125





xk =

[ξkηk

](5.55)


wk =

[wξ

k

wηk

](5.56)


Pk =

[P x

k P x,wk

P w,xk P w

k

](5.57)





xk =

[ξkηk

]P0 =

[P x

0 00 P w

0

](5.58)




125





xk =

[ξkηk

](5.55)


wk =

[wξ

k

wηk

](5.56)


Pk =

[P x

k P x,wk

P w,xk P w

k

](5.57)





xk =

[ξkηk

]P0 =

[P x

0 00 P w

0

](5.58)




125


tackled in two steps. Firstly, a zero order hold is applied to the measurements, secondlythe standard UKF algorithm is properly modified. In more detail, the discrete UKF al-gorithm with an augmented state to consider parameter estimation and process noise isimplemented. The sigma points are recomputed, before propagating them through the ob-servation model. This modification gives the possibility to use the discrete algorithm withthe irregular measurement sampling time of the continuous culture case. Furthermore, theparameter μmax in (5.50) is chosen to be estimated. Finally, despite the fact that a zeroorder hold is used to permit a discrete UKF implementation, the UKF accuracy and speedof convergence are improved with respect to those in the EKF.

� � ��

��

�

��

�

��

�

��

��&('�

��'

'��

��

��

�

�

��&�$��'$$��

� � ��

�

�

�

�

��&('�

��

! ��

��

�

�

��&�$��'&(��"$��##��$��$+��"��'��

Figure 5.2: UKF estimation for simulated batch mode.

Figure 5.2 illustrates the convergence of the UKF in simulated batch mode, for whichconditions (5.51) hold. In this case the nonlinear model (5.44a-5.44b) is discretized atsampling time Ts = 0.5 h and used to simulate the state of the process, starting from

126



� � ��

�%�

!

!%�

�

�%�

�

�%�

��&('�

��'

'��

��

��

�

�

��&�$��'$$��

� � ��

�

�

�

�

��&('�

��

! ��

��

�

�

��&�$��'&(��"$��##��$��$+��"��'��



126



� � ��

�%�

!

!%�

�

�%�

�

�%�

��&('�

��'

'��

��

��

�

�

��&�$��'$$��

� � ��

�

�

�

�

��&('�

��

! ��

��

�

�

��&�$��'&(��"$��##��$��$+��"��'��



126



� � ��

�%�

!

!%�

�

�%�

�

�%�

��&('�

��'

'��

��

��

�

�

��&�$��'$$��

� � ��

�

�

�

�

��&('�

��

! ��

��

�

�

��&�$��'&(��"$��##��$��$+��"��'��



126


0 50 100 150 200 250 300 350 400 4500

0.02

0.04

0.06

0.08

Time [hours]

Fee

d ra

te [l

/h]

(a) Culture medium feeding profile

� ��

�

��

��

��

��

��

� �

��

��

��

��

(b) Measurement of [TIC]

Figure 5.3: Experimental data: input and output of the photobioreactor collected in con-tinuous mode.

initial conditions X0 = 2.44 · 109cell/l, [TIC]0 = 2.55 10−3mole/l. After that, [TIC] iscorrupted by additive Gaussian white noise with standard deviation σ = 0.2 10−3mole/l,and used as measurement for the UKF. Thus, the state is estimated successfully with ex-cellent noise rejection in TIC. The results obtained for continuous cultures are even moreinteresting. Initial conditions are X0 = 1.8 · 109cell/l, [TIC]0 = 4.51 · 10−3mole/l, andin addition real experiment data, presented in Figure 5.3, are used as input to the filters.The EKF designed in Becerra-Celis et al. [2008], the UKF with only state estimation, andthe UKF with joint state and parameter estimation are simulated. The results obtainedare shown in Figure 5.4 and discussed here. In Figure 5.4(a) it is noticeable how bothUKF implementations have faster speeds of convergence than the EKF. In Figure 5.4(b)the state estimation error of the three different approaches are compared, and it is evi-dent how the UKFs give smaller estimation errors. Note that it was possible to compute

127


0 50 100 150 200 250 300 350 400 4500

0.02

0.04

0.06

0.08

Time [hours]

Fee

d ra

te [l

/h]


� ��

�

��

��

��

��

��

� �

��

��

��

��




127


0 50 100 150 200 250 300 350 400 4500

0.02

0.04

0.06

0.08

Time [hours]

Fee

d ra

te [l

/h]


� ��

�

��

��

��

��

��

� �

��

��

��

��




127


0 50 100 150 200 250 300 350 400 4500

0.02

0.04

0.06

0.08

Time [hours]

Fee

d ra

te [l

/h]


� ��

�

��

��

��

��

��

� �

��

��

��

��




127


the state estimation error because experimental data for the biomass, shown with a starin Figure 5.4(a), was available. Moreover the mean squared error (MSE) between thebiomass and its estimate is computed. Table 5.3 shows the MSE index, which is obtainedaveraging the MSE along the entire simulation period. From both figures it is noticeable

Table 5.3: Mean Squared Error IndexEKF UKF UKF with par. est.13.60 6.12 6.12

how the UKF performs better than the EKF, and how the introduction of parameter es-timation in the UKF improves the accuracy of the estimation in the final part (after 300hours), although it slightly reduces the speed of convergence.

Since the parameters are identified from batch experiments, as shown in Becerra-Celis et al. [2008], adding parameter estimation may be useful when the photobioreactoris run in continuous mode. Figure 5.5 shows the evolution of the μmax estimation, theestimated value is compared with the identified value and the UKF error covariance isalso shown.

128






128






128






128


0 50 100 150 200 250 300 350 400 4501

2

3

4

5

6

7

8

9

10

11

Time [hours]

Bio

mas

s X

[109 c

ell/l

]

Experimental dataEKFUKFUKF with parameter est.

(a) Biomass estimate

� � �� ! !� � �� %�

%�

�

�%�

�

� ��!� !"�

��

��

� ��

�

��

(b) State Estimation Error

Figure 5.4: Biomass estimation comparison for continuous cultures.

129


0 50 100 150 200 250 300 350 400 4501

2

3

4

5

6

7

8

9

10

11

Time [hours]

Bio

mas

s X

[109 c

ell/l

]



� ��

�

��

��

�

!��"��

��

��

� ��

�

��



129


0 50 100 150 200 250 300 350 400 4501

2

3

4

5

6

7

8

9

10

11

Time [hours]

Bio

mas

s X

[109 c

ell/l

]



� ��

�

��

��

�

!��"��

��

��

� ��

�

��



129


0 50 100 150 200 250 300 350 400 4501

2

3

4

5

6

7

8

9

10

11

Time [hours]

Bio

mas

s X

[109 c

ell/l

]



� ��

�

��

��

�

!��"��

��

��

� ��

�

��



129


0 50 100 150 200 250 300 350 400 4500.031

0.032

0.033

0.034

0.035

0.036

0.037

Time [hours]

Par

amet

er μ

max

[1/

h]

Identified valueUKF estimate

(a) UKF estimate and identified value of μmax

� ��

��

��

��

��

��

�

��

��

��

��

(b) UKF error covariance of μmax

Figure 5.5: UKF parameter estimate and its covariance for continuous culture.

130


0 50 100 150 200 250 300 350 400 4500.031

0.032

0.033

0.034

0.035

0.036

0.037

Time [hours]

Par

amet

er μ

max

[1/

h]



� ��

��

��

��

��

��

�

��

��

��

��



130


0 50 100 150 200 250 300 350 400 4500.031

0.032

0.033

0.034

0.035

0.036

0.037

Time [hours]

Par

amet

er μ

max

[1/

h]



� ��

��

��

��

��

��

�

��

��

��

��



130


0 50 100 150 200 250 300 350 400 4500.031

0.032

0.033

0.034

0.035

0.036

0.037

Time [hours]

Par

amet

er μ

max

[1/

h]



� ��

��

��

��

��

��

�

��

��

��

��



130

State estimation in Nonlinear Model Predictive Control, UKF advantages

5.4 State estimation in Nonlinear Model Predictive Con-trol, UKF advantages

As discussed in Chapter 2, nonlinear model predictive control has proved to be a suitabletechnique for controlling nonlinear systems, since the simplicity of including constraintsin its formulation makes it very attractive for a large class of applications. A limitationfor the application of NMPC is that at every time step the state of the system is neededfor prediction. However, it is not always possible to measure all the states, thus a filter orobserver may be used. For nonlinear systems a Separation Theorem does not exist, so thateven if the state feedback controller and the observer are both stable, there is no guaranteeof closed-loop nominal stability. Findeisen et al. [2003] give an interesting overview onboth state and output feedback NMPC, and Kolås et al. [2008] focus also on high orderstate estimators and noise model design.

State estimation introduces an extra computational load which can be relevant in thecase of systems with relatively fast dynamics. In this case accurate estimation methodswith low computational cost are desired, for example the Extended Kalman Filter. Clearly,the EKF does not perform well with all nonlinear systems, but its straightforwardness isthe main reason of its popularity.

In this section, a type of locally weakly unobservable system is studied. For this typeof system, we find that the EKF drifts because the system is unobservable at the desiredoperation point. Instead, it is shown how the UKF is used for state estimation in this typeof nonlinear systems, and that it provides a stable state estimate, despite the fact that thesystem is locally unobservable.

5.4.1 An example of locally weakly unobservable systemAs seen in Section 5.1.1, the fundamental requirement for an observer to work properlyis to be associated with an observable system. The following example is constructedsuch that the the observability rank condition derived in Hermann & Krener [1977] is notsatisfied. Thus, the state estimator might have problems once it reaches locally weaklyunobservable regions.

Consider the scalar nonlinear system

x(t) = (0.1x(t) + 1)u(t)

y(t) = x3(t) (5.59)

and apply the rank condition described in Section 5.1.1. Clearly for this system the ob-servability matrix (5.5) is

O(x) =[3x2

](5.60)

131








x(t) = (0.1x(t) + 1)u(t)

y(t) = x3(t) (5.59)


O(x) =[3x2

](5.60)

131








x(t) = (0.1x(t) + 1)u(t)

y(t) = x3(t) (5.59)


O(x) =[3x2

](5.60)

131








x(t) = (0.1x(t) + 1)u(t)

y(t) = x3(t) (5.59)


O(x) =[3x2

](5.60)

131


with rank equal to n = 1 for x �= 0, and rank null for x = 0. Therefore, if we applytheorems 5.1.1 and 5.1.2 we can conclude that (5.59) is locally weakly observable forx �= 0. At x = 0 the system is not locally weakly observable. Thus, in an NMPC contextwe implement both EKF and UKF state estimators. Note that the control goal is to drivethe system to the origin.

Since a discrete time framework is required, we discretize (5.59) using the Eulerapproximation, and a sampling time of 1 second.

xk+1 = xk + (0.1xk + 1)uk + wk

yk = x3k + vk (5.61)

where x is the state, u is the input, y is the measurement, w is the excitation state noise,v is the measurement noise, and the subscript k is the sampling time index. The noisesequences are both assumed to be Gaussian white noise.

Simulation-based results will show that while the EKF fails to estimate the state, theUKF is able to give a stable state estimate. This is due to the implicit structure of theUKF algorithm, in fact no linearization is used. Whereas, when the EKF algorithm isimplemented, the linearization at x = 0 is the origin of EKF failure. In addition the pdfapproximation obtained with the sigma points is more accurate than the EKF one.

Simulation-based results

The NMPC framework (2.16) is applied with the quadratic stage cost function J(x, u) =qx2 + ru2, where the corresponding state weight is q = 2, and the input weight is r = 1.The prediction horizon length is Np = 10. The process and measurement noises haveadditive white Gaussian distributions with zero means and variances σ2

ω = 0.01, σ2v = 0.1,

respectively.For state estimation, the EKF (5.19-5.26) first, and then with the UKF (5.28-5.43)

are implemented. All the simulations are started from an initial condition x0 = 0.5, withinitial state variance P0 = 0.02. The UKF tuning parameters are n = 1, α = 1, β = 2,and κ = 2.

Figure 5.6 shows how the EKF fails. It is possible to note how once both the stateand its estimate reach zero, the estimator is not able anymore to reconstruct the actualstate (Figure 5.6(b)). This is due to the linearization problem. At zero the Kalman gainbecomes zero and the filter is not able to correct the estimate with the future measure-ments. The controller uses the estimate, yielding a zero control signal as soon as the stateestimate is null. As a result the controller is not able to regulate the state anymore and thestate drifts.

In Figure 5.7 the UKF is used instead. It is possible to observe how the filter is ableto estimate the state, and in the meantime there is no drift in the state estimation. Havinga correct state estimate allows the NMPC to regulate the state, achieving its goal.

132




xk+1 = xk + (0.1xk + 1)uk + wk

yk = x3k + vk (5.61)





ω = 0.01, σ2v = 0.1,





132




xk+1 = xk + (0.1xk + 1)uk + wk

yk = x3k + vk (5.61)





ω = 0.01, σ2v = 0.1,





132




xk+1 = xk + (0.1xk + 1)uk + wk

yk = x3k + vk (5.61)





ω = 0.01, σ2v = 0.1,





132


� ��

��

�

�

�

�

�State and its estimate

Timestep

x

�

�

stateest imate

� ��

�

��Control

Timestep

u

(a) System state, state estimate, control input

� � ��

��

��

��

��

�

��

��

��

��

�State and its estimate (zoom)

[T imestep]

x

�

�

stateest imate

(b) State and its estimate: first 40 time steps

Figure 5.6: Simulation-based results: EKF as state estimator.

133


� ��

��

�

�

�

�


Timestep

x

�

�

stateest imate

� ��

�

��Control

Timestep

u


� � ��

��

��

��

��

�

��

��

��

��


[T imestep]

x

�

�

stateest imate



133


� ��

��

�

�

�

�


Timestep

x

�

�

stateest imate

� ��

�

��Control

Timestep

u


� � ��

��

��

��

��

�

��

��

��

��


[T imestep]

x

�

�

stateest imate



133


� ��

��

�

�

�

�


Timestep

x

�

�

stateest imate

� ��

�

��Control

Timestep

u


� � ��

��

��

��

��

�

��

��

��

��


[T imestep]

x

�

�

stateest imate



133


� ��

��

�

��


Timestep

x

�

�

stateest imate

� ��

��

�

��

�State estimation error

Timestep

(a) System state and its estimate, state estimation error

� �� $�

��$�

��$�

��$�

�

�$�

�$�

�$�Control

Timestep

u

� �� $�

��

��$�

�

�$�

�

�$�Measurement

Timestep

y

(b) Control input and system measurement

Figure 5.7: Simulation-based results: UKF as state estimator.

134


� � ��

��

�

��


Timestep

x

�

�

stateest imate

� � ��

��

�

��


Timestep


� �� $�

��$�

��$�

��$�

�

�$�

�$�

�$�Control

Timestep

u

� �� $�

��

��$�

�

�$�

�

�$�Measurement

Timestep

y



134


� � ��

��

�

��


Timestep

x

�

�

stateest imate

� � ��

��

�

��


Timestep


� �� $�

��$�

��$�

��$�

�

�$�

�$�

�$�Control

Timestep

u

� �� $�

��

��$�

�

�$�

�

�$�Measurement

Timestep

y



134


� � ��

��

�

��


Timestep

x

�

�

stateest imate

� � ��

��

�

��


Timestep


� �� $�

��$�

��$�

��$�

�

�$�

�$�

�$�Control

Timestep

u

� �� $�

��

��$�

�

�$�

�

�$�Measurement

Timestep

y



134

Conclusions

5.5 ConclusionsIn this chapter Kalman filtering for state estimation in a model predictive control frame-work is presented. Two examples of state estimation for non nonlinear systems are stud-ied, and the EKF and UKF performances are compared.

In the first example, which can be considered as an extension of Becerra-Celis et al.[2008], it is shown how to improve the estimation performance by using UKF to imple-ment a biomass estimator for a microalgae photobioreactor. In both batch and continuousmode, the approach presented produces faster estimate convergence and better estimateaccuracy. The possibility to easily introduce joint parameter and state estimation, theabsence of linearization, the comparable computational complexity make the UKF an at-tractive estimator for nonlinear systems. The results are validated by comparison withoperational data.

In the second example, an output feedback Nonlinear Model Predictive Control isimplemented for a regulation problem with a particular example from the family of lo-cally weakly unobservable nonlinear systems. For these types of systems the ExtendedKalman Filter may fail to give a correct state estimate, leading to a drift in the state esti-mation. Using a simple but effective state space representation of a particular nonlinearmodel, a set of simulations were carried out to show how the UKF gives a stable stateestimate, despite operating at locally weakly unobservable operating points. Other esti-mation algorithms could be used, for instance Monte-Carlo-based methods, but the choiceof a Kalman-based filter gives the advantage of lower overall computational complexity.

135

Conclusions




135

Conclusions




135

Conclusions




135


136


136


136


136

Chapter 6

Conclusions and recommendations forfurther work

In this chapter, first general conclusions, and then directions for future work are presented.

6.1 ConclusionsThe thesis has focused on MPC performance improvement, and extending the range of ap-plicability for MPC controllers. This is motivated by the success of MPC as an advancedcontrol technique. MPC has a complex structure:

• internal model for plant behavior prediction;• cost function for control quality/index definition;• constraints for representing limitations in the plant, actuators, etc.;• capacity of handling model/plant mismatch and unknown disturbances introducing

feedback action by a receding horizon strategy.All these elements give flexibility and adaptability, but at the same time introduce difficul-ties for theoretical analysis and possible extensions. This neither discourages researchersnor industry, as matter of fact, it is possible to find a vast body of literature and a largenumber of applications of MPC. The results presented in this thesis can be summarizedin three points.

The state dependent input weight in the MPC cost function

This approach introduces smooth gain scheduling functionality into the MPC formulation,and simultaneously improves the conditioning of the Hessian, thus yielding a more robustoptimization formulation. In Chapter 3 this is combined with various representationsof the internal model in the MPC. The best performance and largest operating region isachieved with the state dependent input weight and an LTV model.

137

Chapter 6








137

Chapter 6








137

Chapter 6








137

6. Conclusions and recommendations for further work

The novel PE-MPC formulation

This new formulation is obtained by extending standard MPC to allow for on-line modeladaptation or tuning. It is shown how to extend the constraints set to include the per-sistence of excitation condition, necessary to obtain unbiased parameter estimation. Theparticular method of imposing the PE constraint introduces a new constraint only on thefirst manipulated variable, and the introduction of a backwards-looking horizon alters theMPC controller from memoryless full-state feedback to dynamic full-state feedback. Thisnovel framework allows us to show formally and by examples with FIR and state spacemodels that a periodic control signal arises autonomously, avoiding the necessity of forc-ing this condition by an equality constraint. In general, the formulation given is valid forMulti Input Multi Output systems and general adaptive schemes, yielding a non-convexoptimization problem.

State estimation using UKF

The Unscented Kalman Filter was proposed relatively recently compared to the wellknown Extended Kalman Filter. They can both be used for state estimation of nonlin-ear systems. In UKF there is no need to use linearization procedures, eliminating thenlinearization errors. A set of well specified points, called sigma points, is used insteadto approximate the statistics of the state. This resembles particle filtering, however, theUnscented Transformation allows to choose a relatively small set of ‘particles’ with ob-vious benefits on computational complexity. In fact, the UKF computational burden iscomparable to the EKF one.

Two examples are used to illustrate and compare the performance of the two filters.In the first example a photobioreactor model is introduced, and operational data is alsoused to compare the state estimation error. In the second example a simulation-basedstudy for a locally weakly unobservable nonlinear system is carried out. In both cases,UKF outperforms EKF, especially in the second case where the EKF fails to estimate oncethe system reaches a locally weakly unobservable operating point.

6.2 Further work

This thesis addresses some issues in the areas of MPC, dual control, adaptive control, aswell as state estimation.

Each of these are by themselves large and important problem areas, with many rele-vant research issues. In the following, some remaining topics that are natural extensionsand continuations of the work in this thesis will be proposed.

138







6.2 Further work



138







6.2 Further work



138







6.2 Further work



138

Further work

Cost function with state dependent input weight

This approach could be extended to other applications, where for instance there is a needto increase the region where the controller is able to perform its duties. Verification of theresults in experiments and/or industrial applications would be of interest.


Multi Input Multi Output system implementations yield non-convex optimization prob-lem. A general non-convex optimizer may be used, but it would be interesting to exploitthe PE-constraint structure to obtain efficient optimization implementation. The presentedSingle Input Single Output examples show efficient implementations of the optimiza-tion problems. However, extension to problems with multiple inputs is not necessarilystraightforward. Comparison of different parameter estimation algorithms would also beof interest.

State Estimation issues

For the photobioreactor example, controller implementation coupled with UKF/EKF isthe subsequent step. This will necessarily be left to our collaborators at Supélec.

For the observability issue, the study of larger scale systems would be the naturalsubsequent step. Furthermore, a more theoretical/analytical analysis, aimed at character-izing the locally weakly unobservable systems for which the UKF can work, could beperformed.

Dual Control

It is well known that dual control is in general an intractable problem, but with continuedimprovements in both systems theory and computational power, the area should be re-visited at regular intervals to re-assess what can realistically be achieved.

139

Further work








Dual Control


139

Further work








Dual Control


139

Further work








Dual Control


139


140


140


140


140

References

Allison, B. J., Ciarniello, J. E., Tessier, P. J. C., & Dumont, G. A. (1995). Dual adaptivecontrol of chip refiner motor load. Automatica, 31(8), 1169–1184.

Anderson, B. D. O. & Johnson Jr, C. R. (1982). Exponential convergence of adaptiveidentification and control algorithms. Automatica, 18(1), 1–13.

Aoyama, A., Bell, G., & Walsh, S. P. (1997). Implementation issues in quadratic modelpredictive control. In American Control Conference, Albuquerque, New Mexico.

Bai, E. W. & Sastry, S. S. (1985). Persistency of excitation, sufficient richness and pa-rameter convergence in discrete time adaptive control. System and Control Letters, 6,153–163.

Baquerisse, D., Nouals, S., Isambert, A., Ferreira dos Santos, P., & Durand, G. (1999).Modelling of a continuous pilot photobioreactor for microalgae production. Journal ofBiotechnology, 70(1-3), 335 – 342.

Bar-Shalom, Y. (1981). Stocastic dynamic programming: caution and probing. IEEETransactions on Automatic Control, 26, 1184–1194.

Bar-Shalom, Y. & Tse, E. (1974). Dual effect, certainty equivalence and separation instochastic control. IEEE Transactions on Automatic Control, AC-19, 494–500.

Bar-Shalom, Y. & Tse, E. (1976). Concept and methods in stochastic control. In C. T.Leondes (Ed.), Control and Dynamic Systems, volume 12 (pp. 99–172). New York:Academic Press.

Bayard, D. & Eslami, M. (1985). Implicit dual control for general stochastic systems.Optimal Control Applications and Methods, 6(3), 265–279.

141

References










141

References










141

References










141

References

Becerra-Celis, G., Tebbani, S., Joannis-Cassan, C., Isambert, A., & Boucher, P. (2008).Estimation of microalgal photobioreactor production based on total inorganic carbonin the medium. In 17th IFAC World Congress, Seoul, South Korea.

Bemporad, A. (2006). Model predictive control design: New trends and tools. In 45thIEEE Conference on Decision and Control, San Diego, California.

Bemporad, A., Morari, M., Dua, V., & Pistikopoulos, E. N. (2002). The explicit linearquadratic regulator for constrained systems. Automatica, 38(1), 3–20.

Bemporad, A., Morari, M., & Ricker, N. (2004). Modelpredictive control toolbox for matlab - user’s guide.http://www.mathworks.com/access/helpdesk/help/toolbox/mpc/.

Besançon, G. (1999). A viewpoint on observability and observer design for nonlinearsystems, volume 244 of Lecture Notes in Control and Information Sciences: New Di-rections in nonlinear observer design, (pp. 3–22). Springer Berlin/ Heidelberg.

Birge, J. R. & Louveax, F. (1997). Introduction to Stochastic Dynamic Programming.New York, New York: Springer-Verlag.

Boyd, S. & Vandenberghe, L. (2004). Convex optimization. Cambridge University Press,UK.

Brustad, G. (1991). Dual Control A Monte Carlo Approach. PhD thesis, NorwegianInstitute of Technology (NTH).

Camacho, E. & Bordons, C. (2004). Model Predictive Control. Springer-Verlag, 2ndedition.

Camacho, E. & Bordons, C. (2007). Nonlinear Model Predictive Control: An Intro-ductory Review, volume 358 of Lecture Notes in Control and Information Sciences,assessment and future directions of nonlinear model predictive control 1, (pp. 1–16).Springer Berlin/ Heidelberg.

Chisci, L., Rossiter, J., & Zappa, G. (2001). Systems with persistent disturbances: pre-dictive control with restricted constraints. Automatica, 37, 1019–1028.

142

References












142

References












142

References












142

References

Clarke, D., Mohtadi, C., & Tuffs, P. (1987a). Generalized predictive control-I. The basicalgorithm. Automatica, 23(1), 137–148.

Clarke, D., Mohtadi, C., & Tuffs, P. (1987b). Generalized predictive control-II. Exten-sions and interpretations. Automatica, 23(2), 149–160.

Cutler, C. R. & Ramaker, B. L. (1979). Dynamic matrix control - A computer controlalgorithm. In AIChE Annual Meeting, Houston, Texas.

Cutler, C. R. & Ramaker, B. L. (1980). Dynamic matrix control - A computer controlalgorithm. In Joint Automatic Control Conference, San Francisco, California.

Dasgupta, S., Anderson, B. D. O., & Tsoi, A. C. (1990). Input conditions for continuous-time adaptive systems problems. IEEE Transactions on Automatic Control, 35(1), 78–82.

Dochain, D. (2003). State and parameter estimation in chemical and biochemical pro-cesses: a tutorial. Journal of process control, 13, 801 – 818.

Feldbaum, A. (1960a). Dual control theory (I). Automation and Remote Control, 21(9),1240–1249.

Feldbaum, A. (1960b). Dual control theory (II). Automation and Remote Control, 21(11),1453–1464.

Feldbaum, A. (1961a). Dual control theory (III). Automation and Remote Control, 22(1),3–16.

Feldbaum, A. (1961b). Dual control theory (IV). Automation and Remote Control, 22(2),129–143.

Filatov, N. M. & Unbehauen, H. (2004). Adaptive Dual Control. Springer.

Findeisen, R. & Allgöwer, F. (2002). An introduction to nonlinear model predictive con-trol. In 21st Benelux Meeting on System and Control, Veldhoven, The Netherlands.

Findeisen, R., Imsland, L., Allgöwer, F., & B.A., F. (2003). State and output feedbacknonlinear model predictive control: An overwiev. European Journal of Control, 9(2-3),

143

References














143

References














143

References














143

References

921–936.

Fossen, T. I. (1994). Guidance and Control of Ocean Vehicles. Wiley Publishing.

García, C., Prett, D., & M., M. (1989). Model predictive control: Theory and practice - Asurvey. Automatica, 25(3), 335–348.

Genceli, H. & Nikolaou, M. (1996). New approach to constrained predictive control withsimultaneous model identification. AIChE Journal, 42(10), 2857–2868.

Goodwin, G. C. & Sin, K. S. (1984). Adaptive Filtering Prediction and Control. PrenticeHall.

Green, M. & Moore, J. B. (1986). Persistence of excitation in linear systems. System andControl Letters, 7, 351–360.

Hemerick, J. (1973). Culture methods and growth measurements, (pp. 250–260). Cam-bridge University Press, UK.

Hermann, R. & Krener, A. (1977). Nonlinear controllability and observability. IEEEtransactions on Automatic Control, 22(5), 728–740.

Hovd, M. & Bitmead, R. R. (2004). Interaction between control and state estimation innonlinear MPC. In IFAC Symposium on Dynamics and Control of Process Systems.Boston, Massachusetts.

Hovd, M. & Braatz, R. D. (2001). Handling state and output constraints in MPC usingtime-dependent weights. In American Control Conference, Arlington, Virginia.

Ismail, A., Dumont, G. A., & Backstrom, J. (2003). Dual adaptive control of papercoating. IEEE transactions on control systems technology, 11(3), 289–309.

Julier, S. & Uhlmann, J. (1996). A General Method for Approximating Nonlinear Trans-formations of Probability Distributions. Technical report, Robotics Research Group,Department of Engineering Science, University of Oxford.

Julier, S. & Uhlmann, J. (2004). Unscented filtering and nonlinear estimation. Proceed-ings of the IEEE, 92(3), 401–422.

144

References

921–936.













144

References

921–936.













144

References

921–936.













144

References

Kailtah, T. (1980). Linear systems. Englewood Cliffs, New Jersey: Prentice Hall.

Kall, P. & Wallace, S. W. (1994). Stochastic Programming. New York, New York: JohnWiley & Sons.

Kalman, R. (1960). A new approach to linear filtering and prediction problems. Transac-tions of the ASME - Journal of Basic Engineering, 82(Series D), 35–45.

Kandepu, R., Foss, B. A., & Imsland, L. (2008). Applying the unscented Kalman filterfor nonlinear state estimation. Journal of process control, 18, 753–768.

Khalil, H. K. (2002). Nonlinear systems. Prentice Hall, 3rd edition.

Kolås, S. (2008). Estimation in nonlinear constrained systems with severe disturbances.PhD thesis, Norwegian University of Science and Technlogy (NTNU).

Kolås, S., Foss, B. A., & Schei, T. S. (2008). State estimation IS the real challeng innmpc. In NMPC 2008, International Workshop on Assessment and Future Directionsof Nonlinear Model Predictive Control. Pavia, Italy.

Kothare, M. V., Balakrishnan, V., & Morari, M. (1996). Robust constrained model pre-dictive control using linear matrix inequalities. Automatica, 32(10), 1361–1379.

Kumar, P. & Varaiya, P. (1986). Stochastic Systems: estimation, identification and adap-tive control. Upper Saddle River, New Jersey: Prentice Hall.

Kvasnica, M. (2009). Real-time Model Predictive Control via Multi-Parametric Program-ming: Theory and Tools. Saarbruecken, Germany: VDM Verlag.

Li, D., Qian, F., & Peilin, F. (2005). Research on dual control. ACTA Automatica Sinica,31(1), 32–42.

Maciejowski, J. M. (2002). Predictive control with constraints. Essex, England: PrenticeHall.

Marafioti, G., Bitmead, R. R., & Hovd, M. (2008a). Model predictive control with statedependent input weight: an application to underwater vehicles. In 17th IFAC WorldCongress Seoul, South Korea.

145

References














145

References














145

References














145

References

Marafioti, G., Bitmead, R. R., & Hovd, M. (2010a). Persistently exciting model predictivecontrol. International Journal of Adaptive Control and Signal Processing, (Submitted).

Marafioti, G., Bitmead, R. R., & Hovd, M. (2010b). Persistently exciting model predictivecontrol using FIR models. In International Conference Cybernetics and InformaticsVyšná Boca, Slovak Republic.

Marafioti, G., Olaru, S., & Hovd, M. (2008b). State estimation in nonlinear model pre-dictive control, unscented Kalman filter advantages. In NMPC 2008, InternationalWorkshop on Assessment and Future Directions of Nonlinear Model Predictive Con-trol. Pavia, Italy.

Marafioti, G., Olaru, S., & Hovd, M. (2009a). State Estimation in Nonlinear Model Pre-dictive Control, Unscented Kalman Filter Advantages, volume 384 of Nonlinear ModelPredictive Control Towards New Challenging Applications Series: Lecture Notes inControl and Information Sciences. Springer Berlin/ Heidelberg.

Marafioti, G., Tebbani, S., Beauvois, D., Becerra, G., Isambert, A., & Hovd, M. (2009b).Unscented Kalman filter state and parameter estimation in a photobioreactor for mi-croalgae production. In ADCHEM 2009, International Symposium on Advanced Con-trol of Chemical Processes. Instanbul, Turkey.

Mareels, I. (1985). Sufficiency of excitation. System and Control Letters, 5, 159–163.

Marty, A., Cornet, J., Djelveh, G., Larroche, C., & Gros, G. (1995). A gas phase chro-matography method for determination of low dissolved CO2 concentration and/or CO2

solubility in microbial culture media. Biotechnology Technique, 9(11), 787–792.

Mayne, D., Rawlings, J., Rao, C., & Scokaert, P. (2000). Constrained model predictivecontrol: Stability and optimality. Automatica, 36(36), 789–814.

Mayne, D. Q. (2000). Nonlinear model predictive control: challenges and opportunities,in Nonlinear Model Predictive Control, (pp. 23–44). Birkhauser Verlag.

Moore, J. B. (1982). Persistence of excitation in extended least squares. IEEE Transac-tions on Automatic Control, 28, 69–68.

Naidu, D. S. (2003). Optimal Control Systems. CRC Press.

146

References













146

References













146

References













146

References

Nocedal, J. & Wright, S. (2000). Numerical Optimization. Springer, 2nd edition.

Pluymers, B., Rossiter, J. A., Suykens, J. A. K., & DeMoor, B. (2005). A simple algorithmfor robust MPC. In 16th IFAC World Congress Prague, Czech Republic.

Propoi, A. (1963). Use of LP methods for synthesizing sampled-data automatic systems.Automation and Remote Control, 24, 837–844.

Qin, S. & Badgwell, T. (2003). A survey of industrial model predictive control technology.Control Engineering Practice, 11(7), 733–764.

Rawlings, J. & Mayne, D. Q. (2009). Model Predictive Control: Theory and Design. NobHill Publishing, LLC.

Rawlings, J. & Muske, K. R. (1993). The stability of constrained receding horizon control.IEEE Transactions on Automatic Control, 38(10), 1512–1516.

Richalet, J., Raul, A., Testud, J. L., & Papon, J. (1978). Model predictive heuristic control:Applications to industrial processes. Automatica, 14, 413–428.

Rossiter, J. (2003). Model-based predictive control: a practical approach. New York,New York: CRC Press.

Rugh, W. & Shamma, J. (2000). Research on gain scheduling. Automatica, 36, 1401–1425.

Santos, A. & Bitmead, R. R. (1995). Nonlinear control for an autonomous underwa-ter vehicle (AUV) preserving linear design capabilities. In 34th IEEE Conference onDecision and Control New Orleans, Louisiana.

Scokaert, P. O. M. & Rawlings, J. B. (1999). Feasibility issues in linear model predictivecontrol. AIChE Journal, 45(8), 1649–1659.

Shimizu, K. (1996). A tutorial review on bioprocess systems engineering. Computers andChemical Engineering, 20, 915–941.

Simon, D. (2006). Optimal State Estimation: Kalman, H Infinity, and Nonlinear Ap-proaches. John Wiley & Sons.

147

References














147

References














147

References














147

References

Skogestad, S. & Postlethwaite, I. (2005). Multivariable Feedback Control. Analysis andDesing. John Wiley & Sons.

Sotomayor, O. A. Z., Odloak, D., & Moro, L. F. L. (2009). Closed-loop model re-identification of processes under MPC with zone control. Control Engineering Prac-tice, 17(5), 551–563.

Spivey, B. J., Hedengren, J. D., & Edgar, T. F. (2010). Constrained nonlinear estimationfor industrial process fouling. Industrial & Engineering Chemistry Research, 49, 7824–7831.

Sternby, J. (1976). A simple dual control problem with an analytical solution. IEEEtransactions on Automatic Control, 21(6), 840–844.

Sutton, G. & Bitmead, R. (1998). Experiences with model predictive control applied toa nonlinear constrained submarine. In 37th IEEE Conference on Decision and ControlTampa, Florida.

Tøndel, P., Johansen, T. A., & Bemporad, A. (2003). An algorithm for multiparametricquadratic programming and explicit MPC solutions. Automatica, 39(3), 489–497.

Vada, J. (2000). Prioritized Infeasibility Handling in Linear Model Predictive Control:Optimality and Efficienzy. PhD thesis, Norwegian University of Science and Technlogy(NTNU).

Vada, J., Slupphaug, O., & A., F. B. (1999). Infeasibility handling in linear MPC subjectto prioritized constraints. In 14th IFAC World Congress Beijing, China.

Van Der Merwe, R. & Wan, E. A. (2001). The square-root unscented Kalman filterfor state and parameter estimation. In IEEE International Conference on Acoustics,Speech, and Signal Processing. Salt Lake City, Utah.

Van Der Merwe, R., Wan, E. A., & Julier, S. (2004). Sigma-point Kalman filters fornonlinear estimation and sensor-fusion: Applications to integrated navigation. In AIAAGuidance, Navigation & Control Conference (GNC) Providence, Rhode Island.

Wan, E. A. & Van Der Merwe, R. (2000). The unscented Kalman filter for nonlinearestimation. In Adaptive Systems for Signal Processing, Communications, and Control

148

References












148

References












148

References












148

References

Symposium (pp. 153–158). Lake Louise, Alta. Canada.

Wan, E. A. & Van Der Merwe, R. (2001). Kalman Filtering and Neural Networks, chap-ter 7, (pp. 221–280). Wiley Publishing.

Wittermark, B. (1995). Adaptive dual control methods: An overview. In 5th IFAC sym-posium on Adaptive Systems in Control and Signal Processing Budapest, Hungary.

Wittermark, B. (2002). Adaptive dual control. In Control Systems, Robotics and Automa-tion, Encyclopedia of Life Support Systems (EOLSS), Developed under the auspicesof the UNESCO. [http://www.eolss.net].

Zhang, F. E. (2005). The Schur Complement and Its Applications. Springer.

Zhou, K., Doyle, J. C., & Glover, K. (1996). Robust and Optimal Control. Upper SaddleRiver, New Jersey: Prentice Hall.

149

References







149

References







149

References







149

References

150

References

150

References

150

References

150

Documents

Doctoral theses at NTNU, 2010:235 Giancarlo Marafioti ... · NTNU Norwegian University of Science and Technology Thesis for the degree of philosophiae doctor Faculty of Information