Pomodoro - A novel toolkit for (multiobjective) dynamic

optimization, model based control and estimation

Satyajeet S. Bhonsalea, Dries Telena, Dominique Vercammena, MattiaVallerioa, Jan Hufkensa, Philippe Nimmegeersa, Filip Logista, Jan Van

Impea,∗

aKU Leuven, Department of Chemical Engineering, BioTeC+ & OPTEC, Gebroeders DeSmetstraat 1, B-9000 Ghent, Belgium.

Abstract

This paper presents the software package Pomodoro which includes a collec-tion of algorithms and tools for dynamic optimization. It also introduces theframework for multiobjective problems, model predictive control and stateestimation. Pomodoro is implemented in Python and utilizes CasADi as abackbone to formulate the problem. It uses the orthogonal collocation tech-nique to solve the dynamic optimization problems and efficient third-partysolvers are utilized to solve the resulting nonlinear programs. The designof the software and its main modules are discussed and the user-friendlinessof the software is demonstrated with the help of tutorial examples for eachproblem class. Lastly, the advantages and limitations of this software arediscussed.

Keywords: dynamic optimization, model predictive control, movinghorizon estimation, optimization software

1. Introduction

Optimal control or dynamic optimization, which deals with calculation oftime varying trajectories which optimize a given objective for a system ofdifferential equations, has found a variety of applications in a number of en-gineering disciplines, e.g., trajectory optimization and stabilization control

∗Corresponding AuthorEmail address: jan.vanimpe@cit.kuleuven.be (Jan Van Impe)

Preprint submitted to Expert Systems with Applications May 20, 2016

in aerospace engineering (e.g., Dalin et al., 2015; Gao & Li, 2010); opti-mal path tracking for robots or unmanned aerial vehicles, fuel minimizationin mechanical engineering (e.g., Saerens et al., 2009; Verhelst et al., 2012);or optimal drug administration in biomedical engineering (e.g., Bayon et al.,2016; Naidu et al., 2011). Applications of off-line optimal control in the (bio)-chemical industry vary ranging from calculation of optimal feeding profilesof fed-batch bioreactors (e.g., Liu et al., 2013; Liu, 2009), optimal operationprofiles for batch reactors (e.g Benavides & Diwekar, 2012a,b; Srinivasanet al., 2003) to parameter estimation and optimal experiment design (e.g.,Stamati et al., 2014; Telen et al., 2014; Korkel et al., 2004). Moreover, giventhe contradicting nature of industrial objectives, e.g., reduction of energyusage versus increase in production, multiobjective optimal control is alsobecoming a widely used paradigm in the academia as well as in the indus-try (e.g., Logist & Van Impe, 2012a; Sendın et al., 2009). The solution ofthe multiobjective optimization problem is usually a set of mathematicallyequivalent solutions (cfr. Section 2.2). This allows the decision maker to pickthe solution based on his/her preference. Apart from offline usage, optimalcontrol is used widely in an online setup for control, particularly as ModelPredictive Control, and for estimation as Moving Horizon Estimator. A widevariety of applications of model predictive control as well as moving horizonestimation have been reported in the literature(e.g., Vukov et al., 2015; Lu-cia et al., 2013; Zavala & Biegler, 2009b,c; Nagy et al., 2000; Ricker & Lee,1995).

The last decade has seen a surge in development of new algorithms andcomputational tools that can be used to solve optimal control problems.Moreover, with the growing belief in the open source ideology, many soft-ware packages are available free of cost. Due to the multitude of softwarepackages available, we can only discuss a few of the most common packages.First and foremost, ACADO Toolkit (Houska et al., 2011) is an open sourcesoftware written in C/C++ for automatic control and dynamic optimization.It can be applied to a variety of problems like optimal control, parameterestimation and model predictive control. It includes optimization routinesbased on the sequential quadratic programming (SQP) strategy. Moreover,ACADO Toolkit also allows for easy C code generation (Vukov et al., 2013).ACADO Toolkit has also been extended by Logist et al. (2010) to include mul-tiobjective optimization with various scalarization approaches included in it.ACADO Toolkit however makes use of only the single and multiple directshooting methods to solve these problems. These strategies make use of fast

2

integrators developed for ACADO and already built in the package (Quirynenet al., 2013).

The dsoa package (Fabien, 2010) is also written completely in C/C++ andimplements only the single shooting approach. The optimization problem ishandled by a SQP algorithm. Similarly, MUSCOD-II (Leineweber et al., 2003)is also a C/C++ and Fortran based package which utilizes the multiple shoot-ing approach to solve the optimal control problem. The integrations requiredare handled by a backward difference formula (BDF) or Runge-Kutta basedmethods. It also makes use of the SQP based algorithms to solve the result-ing nonlinear program (NLP). The main advantage MUSCOD-II provides overdsoa is the multiple shooting approach it uses, as single shooting approachesare not often the most efficient for optimal control problems. It is, however,not freely available.

PROPT (TOMLAB Optimization, 2011) is a MATLAB based package whichis commercially available. It solves the optimal control problem using thecollocation approach while the NLP can be solved using various existingsolvers like KNITRO, CONOPT, SNOPT, or CPLEX. DIDO is becoming a widelyused optimal control problem in the aerospace industry. It is also based onMATLAB and makes use of the pseudospectral optimal control theory of Ross& Karpenko (2012). DIDO came especially into prominence after NASA usedit for their Zero-Propellant-Maneuver (Bedrossian et al., 2009) in 2006. Thistool has however not yet caught up with the process chemical industry. An-other MATLAB based tool is dynopt (Cizniar et al., 2005). It also solves theoptimal control problem by the collocation method, while on the optimiza-tion level it utilizes the fmincon function present in MATLAB. Although thispackage is open source, the license for MATLAB still needs to be acquired.

Finally, there are also some packages available which are dedicated to-wards model predictive control apart from ACADO Toolkit. OptCon (Nagy,2008) and NEWCON (Romanenko & Santos, 2007) both make use of the mul-tiple shooting strategy to solve the underlying optimal problem. OptCon isbased on MATLAB and utilizes the HQP solver to solve the NLP, while NEWCON

is based on Fortran. DO-MPC (Lucia et al., 2014) is an interesting tool inparticular for robust model predictive control. It allows for both multipleshooting and collocation approaches to solve the optimal control problemand utilizes interior point based IPOPT to solve the NLP.

We now propose a new software package Pomodoro, which is based onCasADi (Andersson, 2013). Pomodro is equipped to handle all the problemclasses mentioned above, viz., single and multiobjective dynamic optimiza-

3

tion, model predictive control and model based estimation, in one frameworkand is designed to meet the following important criteria.

� Free for academic use: Pomodoro and all the third party softwares itutilizes are freely available to atleast the academic users. This allowsadequate testing of the package and paves way for future modificationbased on the ideas various users may provide. Pomodoro is freely avail-able to all academic users at https://perswww.kuleuven.be/ u0093798/software.php

� Extensibility: As Pomodoro is written using an object-oriented lan-guage Python, it allows other packages to be linked to it easily withoutdisturbing the main skeleton of the existing software.

� User Friendliness: Pomodoro makes use of CasADi’s symbolic algo-rithms. Thus, the process models can easily be implemented by non-experienced programmers. The framework helps user to bypass all thetedious programming which is usually required to solve even a simpledynamic optimization problem. However, an advanced programmerand expert in areas of optimal control can always utilize individualfunctions available in Pomodoro to solve advanced types of problemslike robust optimal control, etc.

This article is structured as follows: First, Section 2 gives a brief intro-duction to the basic elements of (i) dynamic optimization, (ii) multiobjectivedynamic optimization, (iii) model predictive control, and (iv) moving horizonestimation. Section 3 then introduces the developed software tool followedby Section 4 which provides the tutorial examples in Pomodoro. Finally,Section 5 concludes the article by giving the pros and cons of the proposedsoftware and direction for future upgrades.

2. Mathematical background and methods

This section serves to give a mathematical overview of the problem classeswhich can be solved using Pomodoro along with the various techniques whichcan be used to solve them. Section 2.1 discusses dynamic optimization prob-lems, its formulation and solution strategies. Section 2.2 details the ideabehind multiobjective optimal control. Section 2.3 discusses the model pre-dictive controller, while Section 2.4 discusses few of the state estimationtechniques.

4

2.1. Dynamic Optimization

2.1.1. Formulation

The optimal control or dynamic optimization problem (OCP) is describedas

minimizex(·),u(·),p

J =

∫ ξf

ξ0

L(x(ξ),u(ξ),p)dξ +M(x(ξf ),p) (1a)

subject to 0 = f (x(ξ),x(ξ),u(ξ),p, ξ) ; ξ ∈ [ξ0, ξf] (1b)

0 = bi(x(0),p) (1c)

0 ≥ cp(x(ξ),u(ξ),p, ξ) (1d)

0 ≥ ct(x(ξf),u(ξf),p, ξf) (1e)

where x represents the state variables, u the control variables, and p the fixedparameters. The process dynamics are defined by a set dynamic equations fover the interval of the independent variable ξ ∈ [ξ0, ξf]. It has to be notedthat although the optimal control solution is obtained over the interval ofthe independent variable, the final value (ξf) may also be an optimizationvariable, e.g., final time in batch operation. In most cases, f is a systemof ordinary differential equations (ODEs), however, the representation of thedynamic system by differential algebraic equations (DAEs) (e.g., Jockenhovelet al., 2003) or partial differential equations (PDEs) (e.g., Logist & Van Impe,2012a) is also possible. The proposed software is designed to handle largescale ODE models, DAE models with index 1. Systems described by PDEscan also be used if they are relatively small and are discretized spatiallyusing the method of lines of Schiesser (1991). The initial conditions of thesystem are contained in the equality constraints of Eq. 1c, while the Eq.1d-1e represent the path and terminal constraints acting on the system. Theobjective functional can consist of both the Lagrange term L and the MayertermM. It is assumed throughout this paper that all the functions are twicedifferentiable.

2.1.2. Numerical solution

The numerical methods to solve an optimal control problem can be roughlyclassified into two categories: (i) indirect, or “first optimize, then discretize”methods and (ii) direct, or “first discretize, then optimize” methods (Diehlet al., 2006). The indirect method is based on the necessary optimality con-ditions obtained from Pontryagin’s Maximum Principle (Pontryagin et al.,

5

1962) and reformulating the problem such that it leads to a two-point bound-ary value problem which can then be solved using different approaches likesingle shooting, invariant embedding, etc. (Cervantes & Biegler, 2000). Themost common approach, i.e., the direct approach converts the optimal con-trol problem into an nonlinear program (NLP), which is then solved usingappropriate algorithms. The proposed software utilizes the orthogonal col-location of Biegler (2007). There exist other direct methods like the singleshooting and multiple shooting (Bock & Plitt, 1984) which make use of nu-merical integrators to solve the ODE systems. Orthogonal collocation, on theother hand, fully discretizes the state and control variables which act as theoptimization variables in the NLP. Due to the large size of the NLP obtainedby such discretization, dedicated optimization algorithms which exploit thestructure of the system efficiently are required.

2.2. Multiobjective optimal control problems

2.2.1. Formulation

The general multiobjective optimal control problem (MOOCP) formula-tion is similar to the OCP in Section 2.1 with the notable exception that theobjective function now becomes a sequence of functions. For brevity, let the

vector z contain all the optimization variables, i.e., z =[x>, u>, p>

]>. The

optimization problem then becomes,

minimizez

[J1(·), J2(·), . . . Jm(·)] (2)

subject to z ∈ S

where S defines the feasible set which satisfies the constraints of Eq. 1b-1e.For MOOCP, a single solution that optimizes all the (possibly contradicting)objectives cannot be obtained. In contrast, the concept of Pareto optimalset is adopted.Definition A point z* in the feasible space S is Pareto optimal iff theredoes not exist another point z ∈ S, such that Ji(z ≤ Ji(z

*)) for all i andJi(z < Ji(z

*)) for atleast one objective function (Das & Dennis, 1997).

2.2.2. Scalarization methods

Most often scalarization methods are adopted to solve the MOOCP prob-lems. The rationale behind these methods is to convert the MO problem intosingle objective problem by allocating each objective certain weights(w). Byvarying the weights and solving the OCP, an approximation of the Paretoset is obtained.

6

Weighted Sum (WS). This is the most common scalarization technique andbasically combines all the different objectives into a convex weighted sum(Logist et al., 2010). The major intrinsic drawbacks of the WS (Das & Den-nis, 1997) are (i) the non-convex part of the Pareto set cannot be obtained,and (ii) an equal weight distribution does not necessarily lead to an equaldistribution of Pareto points.

Normal Boundary Intersection (NBI). This method proposed by Das & Den-nis (1998), helps to overcome the drawbacks of the WS. The NBI methodproceeds by first constructing the convex hull of individual minima (CHIM),i.e., the set of all convex combinations of the individual minima, and thenconstructing (quasi-)normals to this CHIM. The main reasoning in such aconstruction is that the intersection of the quasi-normal and the boundary ofthe feasible set closest to the utopia point J* (i.e., point containing the min-imum of all the individual minima) is expected to be Pareto optimal. Thus,the optimization problem is reformulated such that it now maximizes thedistance from a point on the CHIM along the quasi-normal without violatingany constraints.

Normalized Normal Constraints (NNC). Messac et al. (2003) proposed thismethod based on a similar normal construction on the CHIM. The normal inthis case, however, acts as an additional inequality constraint to the problem.The idea behind NNC is to minimize a single objective, while reducing thefeasible set by adding the hyperplanes normal to CHIM as constraints. Whenthree or more objectives are present in the optimization problem, it is possiblethat NBI and the NNC methods do not yield the same Pareto points. Theenhanced NNC (ENNC) method proposed by Sanchis et al. (2008) is anextension of the NNC method and is especially useful when more than threeobjectives have to be optimized. Logist & Van Impe (2012b) derived theanalytic relations which linked the solutions of ENNC and NBI, and provedthat these methods generate equivalent Pareto points if the all the additionalinequalities in the ENNC subproblem are active.

For readers interested in the mathematical formulations of these scalariza-tion methods, Miettinen (1998) provides an excellent theoretical backgroundfor multiobjective optimal control.

2.2.3. Vectorization methods

Although in the proposed software we exclusively use scalarization tech-niques, other techniques like vectorization also exist which consider the entire

7

vector of the objectives altogether. This class of techniques generally involvesevolutionary algorithms (e.g., Deb, 2001) in which a population of candidatesolutions evolves slowly towards a Pareto front. Although a large numberof engineering applications have been reported (cfr. Bhaskar et al., 2000;Beiranvand et al., 2014; Montoya et al., 2014; Masehian & Sedighizadeh,2010), these techniques suffer from some inherent drawbacks. Firstly, re-peated function evaluations are required which are time consuming and thuslimit the online application. Such methods also face difficulty in handlinghighly nonlinear constraints. Finally, due to the stochastic nature of theseprocedures low dimensional search space are required. To overcome theseissues, scalarization techniques are usually adopted to solve the underlyingmultiobjective NLP.

2.3. Model predictive control

The general idea behind model predictive control (MPC) is to optimizethe future of a process (by solving an open-loop OCP online) using a processmodel over a certain period (i.e., prediction horizon), and applying the com-puted control input only till a new measurement is available. Once the newmeasurement is acquired, the OCP is solved again to compute the controlinputs. This repeated solution of an OCP on a moving horizon incorporatesa feedback mechanism which can handle various disturbances and uncer-tainties arising out of plant-model mismatch (Rawlings, 2000). An NMPCusually proceeds as follows: at time tk the future dynamics of the processare predicted by the controller (based on the measurements obtained at tk)over a prediction horizon Hp. Based on this prediction the optimal controlinput is computed by optimizing a known objective functional (Eq. 3). Asmentioned before, the true process behavior varies significantly from the pre-dicted behavior due to the plant-model mismatch and inherent disturbances.The feedback required is incorporated by applying the optimal control in-put found at tk only for the time ∆t after which the next measurementis obtained. Using the new measurement obtained at tk+1 = tk + ∆t, theprediction and optimization routine is repeated to find new optimal controlaction and the prediction horizon moves forward. Figure 1 shows the generalprinciple behind the MPC.

8

Figure 1: General principle behind MPC: Based on the measurement at tk an OCP issolved. The solution is applied to the process until new measurement is obtained and thena new OCP is solved.

2.3.1. Formulation

The MPC problem described above can be formulated as follows (Allgoweret al., 2003)

minimizex(·), u(·)

J(·;Hp) =

∫ tk+Hp

tk

L(x(τ),u(τ))dτ +M(tk +Hp)

subject to 0 = f(x,x(τ),u(τ)), x(tk) = x(tk)

u(τ) ∈ U , x(τ) ∈ X ∀τ ∈ [tk, tk +Hp]

(3)

In this formulation, the bar ( ) denotes the state values actually measured orobserved from the plant, where as x, u are the variables computed by solvingthe optimal control problem with the initial condition x. X and U are thefeasible sets for the state and control variables respectively. The variable τrepresents the internal controller time used for the optimal control problem.As the predicted values are bound to be different from the real values, thedistinction between the controller variables (x) and the measured values(x)must be made clear.

This resulting OCP has to solved online using the methods mentioned inSection 2.1.2. The direct single shooting is not usually suitable for real time

9

operations and in most cases, slight modification of multiple shooting (Diehlet al., 2002) and collocation (Zavala & Biegler, 2009a) based techniques areused.

2.4. State estimation

The implementation of MPC requires the entire state vector to be knowna priori. In most situations, the entire state vector cannot directly be mea-sured. Thus in practice, the state vector x must be reconstructed based onthe measurements that are available. For a system with linear dynamics, thisproves to be relatively easy as powerful techniques like the Kalman Filter ex-ist, which give the optimal estimate of the state vector (Kalman, 1960). Fornonlinear processes there exist extensions to the Kalman filter like extendedKalman Filter (Becerra et al., 2001) or unscented Kalman Filter (Julier &Uhlmann, 1996).

2.4.1. Extended Kalman Filter

The Extended Kalman Filter (EKF) computes the state estimate at eachsampling instance by using the Kalman filter on the linearized approximationof the nonlinear process model. If the noise is white Gaussian and a largeneighborhood exists in which the linearization is a reasonable approximationof true model, then the optimal linear estimate will be a good approximationof the nonlinear state estimate. In the proposed software, the First OrderEKF is implemented in which the nonlinear system is linearized around thecurrent state estimate using the first term in Taylors series. If the estimateitself is not close to the true state, then the first order EKF may give highlyerroneous results. This procedure is described in detail in Becerra et al.(2001).

2.4.2. Unscented Kalman Filter

The Unscented Kalman Filter (UKF) is an extension of the so calledunscented transformation to the Kalman filter. The unscented transform isa way to calculate the statistics of a random variable (i.e., the states in thiscase) which is nonlinearly transformed. In UKF, the state distribution isrepresented by a fixed number of deterministically chosen sampling pointscalled the sigma points. These sigma points are then propagated throughthe nonlinear function, and the mean and covariance of this propagation areapproximated. In the current implementation, the sigma points are chosenbased on recommendations from Julier & Uhlmann (1996).

10

2.4.3. Moving Horizon Estimator

The moving horizon estimator (MHE) proposed by Robertson et al. (1996)and later by Rao & Rawlings (2002) is a method which uses an optimiza-tion approach to state estimation. As the estimates are then a solution of aconstrained optimization problem, the estimates never violate the given con-straints. This method is particularly interesting when considered along withthe MPC. The principle behind the MHE is similar to that of the MPC. Atthe current sampling instance k, the MHE considers a set of M measurementsobtained at times [tl, . . . , tk].

Yk = {yl, . . . ,yk},

where l := k −M + 1. Based on this set, an estimate of the states xk isobtained by solving an dynamic optimization problem on the interval [l, k].The horizon then moves forward one step giving a new set of measurements.Thus, MHE can be considered a dual of the MPC problem in the sense thatboth involve the solving a dynamic optimization problem online at everysampling instance and a receding (moving) horizon.

Formulation. The MHE is formulated as an optimization problem as follows:

minimizexj ,p,wj

k∑j=l

‖yj − h(xj,p)‖2Vj +k−1∑j=l

‖wj‖2W + Zl

subject to 0 = f(x,x(τ),u(τ),p,w(τ)), ∀τ ∈ [tl, tk]

xj ∈ X , j = l, . . . , k

wmin ≤ wj ≤ wmax, j = l, . . . , k

(4)

where the estimated measurement vector given by h(·) is evaluated at discretetime intervals j = l, . . . , k− 1. The term w is the process noise which can beconsidered to be the difference between the real process and the model. Theterm Zl summarizes the information contained in the measurements beforej = l. This is called the arrival cost of the MHE problem. Ideally, the exactarrival cost calculated using concepts of dynamic programming (Rao et al.,2003) should be used. However, these costs cannot be expressed analyticallyexcept for the linear unconstrained systems. Thus, various approaches toapproximate and update this arrival cost have been proposed which involveuse of EKF, particle filters or UKF (e.g., Ungarala, 2009; Lopez-Negrete

11

et al., 2009; Qu & Hahn, 2009). The proposed software makes use of thearrival cost update proposed by Kuhl et al. (2011).

The formulation of Eq. 4, without the arrival cost and a receding horizoncan also be used for dynamic parameter estimation problems.

3. Software framework

In this section the design details of the software Pomodoro are discussed.The entire software consists of three main blocks which interact with eachother and with CasADi to generate the solution. These blocks are Pomodoro

itself, Pareto Browser, and SolACE. Each of these packages will be discussedsubsequently.

3.1. Pomodoro

Figure 2: Outline of Pomodoro.Pomodoro utilizes CasADi to formulate the dynamic opti-mization problems. The discretization routines for controls and states inherit from theclass Expression which allows the user to impose constraints on individual collocationpoints.

The basic structure of Pomodoro is outlined in Figure 2. The entirePomodoro tool is written in Python which also acts as the interface of the

12

software. The problem specifications are input by the user in the Python

symbolically as is shown in Section 4. Pomodoro then contains the routineswhich convert these symbolic equations into an NLP. Following the orthog-onal collocation strategy, the state variables are discretized by third orderpolynomials using Radau collocation points (in the class LagrangeStates),while the control variables are discretized as piecewise constant(in the classPiecewiseConstant). These classes inherit properties from their base classExpression. Expression allows the user to get and pass information onany individual collocation (or discretization) point, or at any time instance.This is a major advantage as now a constraint can be imposed easily at anytime instance in the problem. This is helpful especially during parameterestimation problems when equidistant sampling times are not possible.

The class Problem acts like a buffer to collect all the information aboutthe (discretized) states and controls, system ODEs, objective functions, andconstraints. The problem is then passed to the Solver class which initializesthe NLP solver, and on command the solve problem. The default solver isset to be IPOPT (Wachter & Biegler, 2006), although other solvers like WORHP

(Buskens & Wassel, 2013) and CasADi’s inbuilt SQP solver can also be used.It is, however easily possible to extend this class to include other solversavailable commercially or as open source. CasADi provides the solver withexact jacobian and hessian information by efficiently exploiting the automaticdifferentiation techniques. The solution of the NLP is passed back to theclass Problem through the Expression routine, which makes the numericalsolution at any individual collocation point available.

3.2. Pareto Browser

Pareto Browser is the multiobjective arm of the software package. Itutilizes the scalarization techniques described in Section 2.2 to solve themultiobjective optimization problem and generate a Pareto front. The mainadvantage of package Pareto Browser is its novel interactive visualizationframework.

The problem class for multiobjective optimization, MOProblem, inheritsfrom the the base class Problem of Pomodoro. MOProblem scalarizes the mul-tiobjective problem and passes it on to its base class. The scalarized problemis solved for varying weights, which are generated automatically or are pro-vided by the user. The solver class for multiobjective problem is called theMultiobjective and inherits from the main Sovler class. Multiobjectiveuses the Solver to solve the OCP repeatedly at the different weights. The

13

results can be visualized directly or be passed on to the interactive tool –Pareto Browser.

The class ParetoBrowser, consists of the interactive tool through whichthe trajectories of the states and controls can be visualized for differentweights. It is possible to add or remove points from the Pareto front ifthey do not meet the users’ requirements. Moreover, a Pareto front can bevisualized for two or three objectives. For problems of higher dimensions,alternate visualization techniques like Parallel Coordinates (Inselberg, 2009)are used. The class interacts interactively with other classes like Solver toobtain state and control trajectories at any required Pareto point. The mainidea of the Pareto Browser in itself has been described in Vallerio et al.(2015).

3.3. SolACE: Solver for Advanced Control and Estimation

SolACE, the model predictive control and state estimation toolkit, is builtupon the Pomodoro. The package currently consists of various routines whichcan be used to introduce the controller and/or the estimator. The models canbe intuitively introduced as shown in Section 4.3, and the controller/estima-tor routines contain the required code to automatically formulate the controlproblem. The problem class for MPC, MPCProblem partially inherits from thebase class Problem, which collects all the information required to formulatethe OCP. This OCP is converted into an NLP and solved by Pomodoro asdescribed previously in Section 3.1. SolACE assumes that the measurementsare available after equal intervals, and control solution obtained is used tosimulate plant behavior over this interval. MPCProblem also contains theroutines to introduce plant models. Within the simulation environment, it ispossible to introduce model-plant mismatch by defining different models forthe plant and the controller. The plant simulation is carried out by integrat-ing the system ODEs for the obtained control solution using the Sundials

(Hindmarsh et al., 2005) toolkit. The solution of the plant simulation is usedas the initial value to resolve the OCP. By default, it is assumed that all thestates are measured and there is no measurement noise present. For a morerealistic simulation, measurement noise can be incorporated into the plantsimulation and a separate measurement vector can be defined. To estimatethe state vector from the measurements the Estimator class is used.

A beneficial feature of SolACE is that three different techniques of es-timation - Extended Kalman Filter, Unscented Kalman Filter and MovingHorizon Estimation are already implemented and can directly be used. By

14

defining which technique has to be used the Estimator class inherits proper-ties from a relevant class. For EKF and UKF, the class acts as an integratorusing the Sundials suite, whereas for MHE, the class inherits from the baseclass Problem which defines an OCP. All the three routines can be simulatedthrough the class MPCSolve, which sequentially works as follows: (i) from agiven initial condition the OCP for the MPC is solved, (ii) by consideringonly a part of control trajectory the plant is simulated by integrating theODEs, and (iii) using the solution of the integration as measurements, theestimators generate the full state vector which then acts as the initial valuefor the OCP. Moreover, any other custom estimator can be easily added tothe Estimator by defining a separate function and passing it over.

4. Tutorial examples

In this section, Pomodoro is demonstrated using a few test cases. Thefirst test case considers a simple time optimal control problem with a singleobjective. In the second case, the same case is extended to consider multipleobjective to demonstrate the Pareto Browser. The third case demonstratesthe closed-loop simulations under MPC, with an active state estimation rou-tine. More case studies can be found in the repository obtained along withPomodoro.

4.1. Single objective optimal control problem

In this section an optimal control problem of the form given by Eq. 5 isset up and solved.

minimizex(·),u(·)

tf

subject todx1dt

= x2

dx2dt

= u

x(0) = [0 0]

x(tf ) = [400 0]

x2(t) ≤ 40

tf ≤ 100

(5)

The problem is to move a car from its initial location (x1(0) = 0) toa specific location (x1(tf ) = 400) in minimum time but without getting

15

Code Listing 1: Implementation of OCP defined in Eq. 5

1 import numpy as NP2 from pomodoro . problem . problem import Problem3 from pomodoro . s o l v e r . s o l v e r 2 import So lver24 from pomodoro . d i s c s . exp r e s s i on import Express ion5 from casad i import *

6 import matp lo t l i b . pyplot as p l t7 prob = Problem ( )8 # Set up Problem Class9 tend = prob . setTimeRange ( 0 . 0 , ’ f r e e ’ , 0 . 1 , 1 0 0 , 2 5 . 0 )

10 # Spec i f y time range11 x = prob . addStates ( 2 , [ 0 . 0 , 0 . 0 ] , [ 4 0 0 . 0 , 4 0 . 0 ] )12 # Def ine s t a t e s : number , lower bound , upper bound13 u = prob . addControls ( 1 , [ − 5 . 0 ] , [ 5 . 0 ] )14 # Def ine c o n t r o l s : same as s t a t e s15 rhs = Express ion (SX. z e r o s (2 ) )# Def ine ODEs16 rhs [ 0 ] = x [ 1 ] # o f the17 rhs [ 1 ] = u [ 0 ] # car model18 prob . addOde(x , rhs ) # Pass ODEs to the problem19 f 2 = tend /10 #Object ive Function20 prob . addConstra ints ( x (0 ) , [ 0 . 0 , 0 . 0 ] )21 #I n i t i a l Condit ions as Equal i ty Const ra int s22 prob . addConstra ints ( x(−1) , [ 4 0 0 . , 0 . ] ) # Terminal Constra int23 prob . addObject ive ( f 2 ) # Declare o b j e c t i v e24 s o l v e r = So lver2 ( prob , p r i n t l e v e l =5)25 # Set up s o l v e r26 s o l v e r . s o l v e ( ) # Solve OCP27 x . p l o t ( ) # Plot S ta t e s28 u . p l o t ( ) # Plot Contro l s

a speeding ticket (i.e., velocity less than 40). The car is initially at rest(u(0) = 0), and as infinite acceleration is never possible, the acceleration isbound between −5 ≤ u(t) ≤ 5. The corresponding implementation inPomodoro is given in Code Listing 1. The default setting of Pomodoro is todiscretize the states and controls using 50 collocation points, and to use IPOPTto solve the resulting NLP. Moreover, for this case no initialization is requiredto solve the optimization problem. Pomodoro automatically initializes theproblem at values between the box constraints. However, if required thestates and control can be initialized in two ways: (i) if initialization value isconstant throughout, it can be provided while defining the state, and (ii) if

16

(a) State Trajectories (b) Control Trajectory

Figure 3: State and control trajectories for the single objective car problem as visualizedby Pomodoro

the initialization values vary between collocation points, they can be providedin form of a text (.txt) file.

The Solver routine also by default uses IPOPT. The matplotlib libraryof Python provides with visualization tools that are used through out. It canbe noticed that the syntax in Code Listing 1 is intuitive from the problemdescription. Figure 3 shows the state and control trajectories calculated byPomodoro. It is clear from the figure that the the problem is symmetric, i.e.,the system behaves exactly opposite in the second half to what it did in thefirst.

4.2. Multiobjective optimal control problem

The multiobjective problem demonstrated in this section is a simple ex-tension of the previous case. Here, a second objective is added which is toreduce the fuel usage (mathematically, this can be represented as minimiza-tion of the integral of the control actions over time interval). It follows thatthe two objectives are contradicting and reducing the value of one objectivefunction value will increase the other. On account of the observed symmetry,it is assumed that in the second half of the time interval, the car behavesexactly opposite to what it did till this halfway point. Thus the problem issimplified by considering only the half of the entire time interval. The CodeListing 2 shows the implementation of the problem in Pomodoro. Again,it can clearly be noticed that the syntax is similar to the single objective

17

Code Listing 2: Implementation of Multiobjective OCP

1 import numpy as NP2 from pomodoro . problem . moproblem import MOproblem3 from pomodoro . s o l v e r . m u l t i o b j e c t i v e import M u l t i o b j e c t i v e4 from pomodoro . d i s c s . exp r e s s i on import Express ion5 from pomodoro . graphgenerator . paretobrowsers import

ParetoBrowsers6 from casad i import *

7 prob = MOproblem(11 , ’NBI ’ )8 # Def ine problem : Number o f pareto points , s c a l a r i z a t i o n method9 tend = prob . setTimeRange ( 0 . 0 , ’ f r e e ’ , 0 . 1 , 5 0 , 2 5 . 0 )

10 # Spec i f y time range , the end time i s opt imiza t i on parameterhere

11 x = prob . addStates ( 2 , [ 0 . 0 , 0 . 0 ] , [ 2 0 0 . 1 , 4 0 . 0 ] )12 # Def ine Sta t e s13 u = prob . addControls ( 1 , [ 0 . 0 ] , [ 5 . 0 ] )14 # Def ine Contro l s15 rhs = Express ion (SX. z e r o s (2 ) ) # Implementation o f16 rhs [ 0 ] = x [ 1 ] # the ODEs17 rhs [ 1 ] = u [ 0 ] # f o r car model18 prob . addOde(x , rhs ) # Pass ODE to problem19

20 f 1 = sum(u [ 0 ] ( ’ c o l l ’ ) ) # The two21 f 2 = tend /10 # o b j e c t i v e s22

23 prob . addConstra ints ( x (0 ) , [ 0 . 0 , 0 . 0 ] )24 # I n i t i a l Values as e q u a l i t y c o n t r a i n t s25 prob . addConstra ints ( x [0 ] ( −1) ,200)26 # Terminal e q u a l i t y c o n s t r a i n t s27 prob . addMult ip l eObject ives ( [ f1 , f 2 ] )28 # Provid ing problem mul t ip l e o b j e c t i v e s29 s o l v e r = M u l t i o b j e c t i v e ( prob , p r i n t l e v e l =5)30 # I n i t i a l i z e the s o l v e r31 s o l v e r . s o l v e ( ) # Solve32 p lo t = ParetoBrowsers ( prob , s o l v e r )33 # I n i t i a l i z e Pareto Browser ( V a l l e r i o et . a l 2015)34 p lo t . p l o t ( )

case and it is easy to switch between the three scalarization techniques. ThePareto Browser allows the user to interactively add or remove Pareto pointsbased on his choice and visualize the state and control trajectories at these

18

points. Further details about the Pareto Browser and its features, can befound in Vallerio et al. (2015). A screen shot of this interactive tool is shownin Figure 4

19

Figure 4: Screenshot of the Pareto Browser. On the left, the Pareto front is depicted, F1 being the fuel minimization objectiveand F2 the end time minimization objective (divided by 10). The top right figure illustrates the state profiles, while thebottom right figure depicts the control profiles at the Pareto points highlighted in the Pareto front

20

4.3. Model predictive control with state estimation

Code Listing 3: Implementation of NMPC and MHE

17 MPC = MPCproblem(300 , t o t a l p l a n t =3000) # Def ine MPC Problem18 x = MPC. addCont ro l l e rS ta t e s ( 4 , [ 0 . 0 , 0 . 0 , 5 0 . 0 , 5 0 . 0 , −1 . 0 e06

] , [ 1 0 . 0 , 1 0 . 0 , 2 5 0 . 0 , 2 5 0 . 0 , 1 . 0 e06 ] , [ 2 . 1 4 , 1 . 0 9 , 1 1 4 . 2 , 1 1 2 . 9 , 0 . 0 ] )# Def ine c o n t r o l l e r s t a t e s

19 u = MPC. addContro l l e r Inputs ( 2 , [ 3 . 0 , − 9 0 0 0 . 0 ] , [ 3 5 . 0 , 0 . 0 ] )# Def inec o n t r o l l e r inputs

20 rhs = Express ion (SX. z e r o s (4 ) )21 cA = x [ 0 ] ; cB = x [ 1 ] ; theta = x [ 2 ] ; thetaK = x [ 3 ]22 k1 = k10*exp (E1/(273 .15 +theta ) ) ; k2 = k20*exp (E2/(273 .15 +theta )

)23 k3 = k30*exp (E3/(273 .15 +theta ) )24 rhs [ 0 ] = (1/ tph ) *(u [ 0 ] * ( cA0−cA) − k1*cA − k3*cA*cA)25 rhs [ 1 ] = (1/ tph ) * (− u [ 0 ] * cB + k1*cA − k2*cB)26 rhs [ 2 ] = (1/ tph ) *(u [ 0 ] * ( theta0−theta ) − (1/( rho*Cp) ) *( k1*cA*H1

+ k2*cB*H2 + k3*cA*cA*H3)+(kw*AR/( rho*Cp*VR) ) *( thetaK −theta ))

27 rhs [ 3 ] = (1/ tph ) * ( ( 1/ (mK*CPK) ) *(u [ 1 ] + kw*AR*( theta−thetaK ) ) )28 MPC. addControllerODEs (x , rhs ) # Def ine c o n t r o l l e r ODEs38 f = sum( S00*X0( ’ c o l l ’ ) *X0( ’ c o l l ’ ) + S11*X1( ’ c o l l ’ ) *X1( ’ c o l l ’ ) +

S22*X2( ’ c o l l ’ ) *X2( ’ c o l l ’ ) + S33*X3( ’ c o l l ’ ) *X3( ’ c o l l ’ ) + S44*U1( ’ c o l l ’ ) *U1( ’ c o l l ’ ) + S55*U2( ’ c o l l ’ ) *U2( ’ c o l l ’ ) )

39 MPC. addContro lObject ive ( f ) # Tracking Object ive41 x1 = MPC. addPlantStates (4 ) # Def ine Sta t e s f o r Plant42 u1 = MPC. addPlantInputs (2 ) # Def ine inputs f o r p lant56 MPC. addPlantODEs ( x1 , rhs1 ) # Def ine Plant ODEs57 MPC. s e t I n i t C o n d i t i o n ( x0 ) # Def ine I n i t i a l Condit ion58 MPC. addMeasurementNoise ( [ 2 , 3 ] , [ 2 , 2 ] ) # Incorpora te Noise64 e s t = Estimator ( ’mhe ’ , [ 2 , 3 ] ,V, P00 ) # Def ine e s t imator : type ,

measured s ta t e s , var iance , i n i t i a l covar iance matrix65 e s t . de f ineHor i zon ( 2 0 , 1 0 . ) # For MHE, d e f i n e l ength o f hor izon ,

sampling i n t e r v a l84 e s t . addMeasurements (yE)85 e s t . addEstimatorODEs ( x2 , rhs2 )86 s o l v e r = MPCsolve (MPC, Estimator = e s t ) # Def ine MPC problem with

es t imator87 s o l v e r . s o l v e ( )88 s o l v e r . p l o t S t a t e s ( )#89 s o l v e r . p l o tCont ro l s ( )#90 s o l v e r . plotMeasurements ( )

21

For demonstration of the package, the Van de Vusse reactor (e.g., Bonillaet al., 2010) is selected as a simple case study. The Van de Vusse (VDV)rector consists of a continuous stirred tank reactor in which two competingexothermic reactions take place. The heat generated by these reactions isremoved from the reactor through an external cooling jacket. Operating thereactor at different conditions leads to different yields of the products. Thus,it is necessary to operate the reactor at a constant set-point value which willachieve the optimal yield of the desired product. The objective of the MPCis to track the given steady state set point effectively. The process dynamicsof the reactor are given by Eq. 6. The parameter values for the model arepresented in the Table 1.

dCadt

=F

Vr(Ca0 − Ca)− k1Ca − k3C2

a

dCbdt

=F

Vr(−Cb) + k1Ca − k2Cb

dθ

dt=F

Vr(θ0 − θ) +

kwArρCpVr

(θk − θ)−1

ρCp(R1 +R2 +R3)

dθkdt

=1

mkCpk(Qk + kwAr(θ − θk))

(6)

Furthermore, it is assumed that the only measurements available are fromthe two temperature sensors for the reactor and the jacket. Furthermore, itis assumed that these sensors introduce additive Gaussian noise defined bystandard deviation σ2 = ( 2 K 0

0 2 K ). The objective of the controller is toachieve a steady state set point obtained from steady state optimization.The objective function is formulated as a quadratic tracking equation givenin Eq. 7

J =

∫ t+Hp

t

[x− xs]>Q [x− xs] + [u− us]

>R [u− us] dt (7)

where x = [Ca, Cb, θ, θk], i.e., the concentrations of A and B, and the

temperatures in the reactor and the jacket are the states. u =[FVr, Qk

], i.e.,

the dilution rate and heat removal rate, are the manipulated variables. The

22

matrices Q and R are the weighting matrices given as,

Q =

0.2 0 0 00 1.0 0 00 0 0.5 00 0 0 0.2

, R =

(0.5 00 5e− 7

)

Finally, the steady state values that have to be tracked are given by xs =[2.14 mol/l, 1.04 mol/l, 114.2 ◦C, 112.9 ◦C] and us =

[14.19 h−1, −1113.5 kJ/h

].

Table 1: Van de Vusse Reactor Parameter Values

Parameter Value Parameter Value

k10 1.287 · 1012 h−1 ρ 0.9342 kgl

k20 1.287 · 1012 h−1 Cp 3.01 kJkJ·K

k30 9.043 · 109 h−1 kw 4032 kJh·m2·K

E1 −9758.3 J/mol AR 0.215 m2

E2 −9758.3 J/mol VR 10 lE3 −8560 mK 5 kgH1 4.2 kJ

molCPK 2.0 kJ

kJ·KH2 −11 kJ

molcA0 5.1 mol

l

H3 −41.85 kJmol

θ0 104.9 ◦C

The Van de Vusse reactor is optimized with the objective given in Eq. 7,constrained by the process model Eq. 6, for 3000 s over 300 cycles. This givesa sampling time of 10s. The open-loop OCP consider a horizon of 100s topredict and obtain the control values for the process. The objective is to trackthe given reference steady state values start from the initial conditions. Theonly known measurement are the temperatures of the reactor and the jacket.A MHE formulation is implemented to estimate the two concentration. TheMHE utilizes a horizon of 10 measurements and the information before thatis approximated in the arrival cost update Kuhl et al. (2011).

The most important parts of implementation of the above problem areshown in Code Listing 3. The entire code can be found in the examplerepository that is included in the Pomodoro package. Figures 5-6 illustratethe closed loop simulation of the Van de Vusse reactor with the different

23

Figure 5: State trajectories for the MPC problem. In red are the set points, and ingreen are the plant measurements disturbed by measurement noise. In blue are the statetrajectories from the controller

Figure 6: Control trajectories for the MPC problem. In red are the set points, and in blueare the trajectories from the controller

24

estimation methods. It can be observed from Figure 5 that, even when justthe two temperatures are measured, the full state can be almost accuratelybe reconstructed and the NMPC manages to track the steady state values.

5. Conclusions and future work

In this paper, an free-for-academic use software Pomodoro has been pre-sented. The software can be used for single and multiple objective optimalcontrol problems along with model predictive control and state estimation.A brief description of each of the packages within the toolkit was providedalong with three tutorial examples. One of the advantages Pomodoro pro-vides is that the constraints can be imposed at any individual collocationpoint. Moreover, from an estimation perspective, techniques such as EKFand UKF are already integrated, while others can be easily incorporated.The CasADi skeleton of Pomodoro allows exact calculation of jacobians andhessian which aids in the smooth running of optimization routines. Pomodoroprovides an option between various solvers for optimization. Currently,IPOPT, WORHP are the external optimization solvers linked. The major draw-back of Pomodoro, is that it is not self-contained and makes extensive use ofthird-party softwares like CasADi, IPOPT, WORHP, HSL, qpOASES.

Pomodoro has been used internally in the authors’ division for a varietyof problems. Vallerio et al. (2016) utilize Pomodoro to optimize the chemi-cal vapor deposition reactor in an robust and multiobjective framework, inVercammen et al. (2016), the MHE framework is utilized to estimate thedynamic fluxes in metabolic networks, whereas Telen et al. (2014) utilizeit for robust optimal experiment design studies. All these studies lead tolarge problems (e.g., robust OCP formulation of chemical vapor depositionreactor leads to ODE system with 50 states) which were efficiently solved byPomodoro. Other studies which make extensive use of Pomodoro are Nim-megeers et al. (2016); Vallerio et al. (2015); Telen et al. (2015); Vercammenet al. (2014), etc.

Pomodoro currently includes all the functionalities described in this paper.The on going development is related to implementation of multiple shoot-ing algorithms in Pomodoro, along with links to other optimization routines.From an NMPC perspective, the real time algorithms both in multiple shoot-ing and collocation will be implemented. Various robustification techniqueswhich make use of sigma points, polynomial chaos expansions are being in-vestigated and it is planned to integrate these with Pomodoro and Pareto

25

Browser. Moreover, the ultimate aim of Pomodoro (especially SolACE) is tobecome an interface between the user and the real plant. To make it moreuser friendly and industry-oriented, it is planned to link Pomodoro to existingflowsheet simulators like ASPEN Plus or gProms. This will allow the user toimport dynamic models directly by building flowsheets and thus reducing themodeling effort to a bare minimum.

6. Acknowledgments

The research was supported by: PFV/10/002 (OPTEC), PT/10/035.FWO KAN2013 1..189.13. FWO-G.0930.13 and BelSPO: IAP VII/19 (DYSCO).SSB holds a Baekeland PhD grant (03/2016 - 03/2020) from the Agency forInnovation through Science and Technology in Flanders (IWT). DT holdsPDM grant 2015/134 from KU Leuven. JVI held the chair Safety Engineer-ing sponsored by Belgian Chemistry and Life Sciences Federation essencia.

References

Allgower, F., Findeisen, R., & Ebenbauer, C. (2003). In UNESCO Encyclope-dia of Life Support Systems (EOLSS) chapter Nonlinear Model PredictiveControl. EOLSS Publishers Co. Ltd. volume XI.

Andersson, J. (2013). A General-Purpose Software Framework for DynamicOptimization. PhD thesis Arenberg Doctoral School, KU Leuven.

Bayon, L., Otero, J., Suarez, P., & Tasis, C. (2016). New developments inthe application of optimal control theory to therapeutic protocols. Math-ematical Biosciences , 272 , 34 – 43.

Becerra, V., Roberts, P., & Griffiths, G. (2001). Applying the extendedKalman filter to systems described by nonlinear differential-algebraic equa-tions. Control Engineering Practice, 9 , 267 – 281.

Bedrossian, N., Bhatt, S., Kang, W., & Ross, I. (2009). Zero-propellantmaneuver guidance. Control Systems, IEEE , 29 , 53–73.

Beiranvand, V., Mobasher-Kashani, M., & Bakar, A. A. (2014). Multi-objective PSO algorithm for mining numerical association rules withouta priori discretization. Expert Systems with Applications , 41 , 4259 – 4273.

26

Benavides, P. T., & Diwekar, U. (2012a). Optimal control of biodiesel pro-duction in a batch reactor: Part I: Deterministic control. Fuel , 94 , 211 –217.

Benavides, P. T., & Diwekar, U. (2012b). Optimal control of biodiesel pro-duction in a batch reactor: Part II: Stochastic control. Fuel , 94 , 218 –226.

Bhaskar, V., Gupta, S., & Ray, A. (2000). Applications of multiobjectiveoptimization in chemical engineering. Reviews in Chemical Engineering ,16 , 1 – 54.

Biegler, L. T. (2007). An overview of simultaneous strategies for dynamic op-timization. Chemical Engineering and Processing: Process Intensification,46 , 1043–1053.

Bock, H., & Plitt, K. (1984). A multiple shooting algorithm for direct solutionof optimal control problems. In Proceedings of the 9th IFAC world congress,Budapest (pp. 243–247). Pergamon Press.

Bonilla, J., Diehl, M., Logist, F., De Moor, B., & Van Impe, J. (2010).A convexity-based homotopy method for nonlinear optimization in modelpredictive control. Optimal Control Applications and Methods , 31 , 393–414.

Buskens, C., & Wassel, D. (2013). Modeling and Optimization in SpaceEngineering. chapter The ESA NLP Solver WORHP. (pp. 85–110). NewYork, NY: Springer New York.

Cervantes, A. M., & Biegler, L. T. (2000). Optimization Strategies for dy-namic systems. Encyclopedia of Optimization, (pp. 2847–2858).

Cizniar, M., Salhi, D., Fikar, M., & Latifi, M. A. (2005). A MATLAB packagefor orthogonal collocations on finite elements in dynamic optimisation.In J. D. J. Mikles, M. Fikar (Ed.), Proceedings of the 15th InternationalConference Process Control ’05 (pp. 058–1 – 058–7). Strbske Pleso, HighTatras, Slovakia.

Dalin, Y., Bo, X., & Youtao, G. (2015). Optimal strategy for low-thrust spiraltrajectories using lyapunov-based guidance. Advances in Space Research,56 , 865 – 878.

27

Das, I., & Dennis, J. (1998). Normal-Boundary Intersection: A new methodfor generating the Pareto surface in nonlinear multicriteria optimizationproblems. SIAM Journal on Optimization, 8 , 631–657.

Das, I., & Dennis, J. E. (1997). A closer look at drawbacks of minimiz-ing weighted sums of objectives for pareto set generation in multicriteriaoptimization problems. Structural optimization, 14 , 63–69.

Deb, K. (2001). Multi-Objective Optimization Using Evolutionary Algo-rithms . New York, NY, USA: John Wiley & Sons, Inc.

Diehl, M., Bock, H., Diedam, H., & Wieber, P.-B. (2006). Fast direct multipleshooting algorithms for optimal robot control. In M. Diehl, & K. Mombaur(Eds.), Fast Motions in Biomechanics and Robotics (pp. 65–93). SpringerBerlin Heidelberg volume 340 of Lecture Notes in Control and InformationSciences .

Diehl, M., Bock, H. G., Schloder, J. P., Findeisen, R., Nagy, Z., & Allgower,F. (2002). Real-time optimization and nonlinear model predictive control ofprocesses governed by differential-algebraic equations. Journal of ProcessControl , 12 , 577 – 585.

Fabien, B. C. (2010). dsoa: The implementation of a dynamic system opti-mization algorithm. Optimal Control Applications and Methods , 31 , 231–247.

Gao, Y., & Li, X. (2010). Optimization of low-thrust many-revolution trans-fers and Lyapunov-based guidance. Acta Astronautica, 66 , 117 – 129.

Hindmarsh, A. C., Brown, P. N., Grant, K. E., Lee, S. L., Serban, R., Shu-maker, D. E., & Woodward, C. S. (2005). SUNDIALS: Suite of nonlinearand differential/algebraic equation solvers. ACM Transaction on Mathe-matical Software, 31 , 363–396.

Houska, B., Ferreau, H., & Diehl, M. (2011). ACADO Toolkit - an open-source framework for automatic control and dynamic optimization. Opti-mal Control Applications and Methods , 32 , 298–312.

Inselberg, A. (2009). Parallel Coordinates . (1st ed.). Springer-Verlag NewYork.

28

Jockenhovel, T., Biegler, L. T., & Wachter, A. (2003). Dynamic optimizationof the tennessee eastman process using the optcontrolcentre. Computers& Chemical Engineering , 27 , 1513 – 1531.

Julier, S., & Uhlmann, J. (1996). A General Method for ApproximatingNonlinear Transformations of Probability Distributions . Technical ReportTechnical report, Robotics Research Group, Department of EngineeringScience, University of Oxford.

Kalman, R. E. (1960). A New Approach to Linear Filtering and PredictionProblems. Transactions of the ASME Journal of Basic Engineering , (pp.35–45).

Korkel, S., Kostina, E., Bock, H. G., & Schloder, J. P. (2004). Numericalmethods for optimal control problems in design of robust optimal exper-iments for nonlinear dynamic processes. Optimization Methods and Soft-ware, 19 , 327–338.

Kuhl, P., Diehl, M., Kraus, T., Schloder, J. P., & Bock, H. G. (2011). Areal-time algorithm for moving horizon state and parameter estimation.Computers & Chemical Engineering , 35 , 71 – 83.

Leineweber, D. B., Schafer, A., Bock, H. G., & Schloder, J. P. (2003). Anefficient multiple shooting based reduced SQP strategy for large-scale dy-namic process optimization: Part II: Software aspects and applications.Computers & Chemical Engineering , 27 , 167 – 174.

Liu, C. (2009). Optimal control for nonlinear dynamical system of microbialfed-batch culture. Journal of Computational and Applied Mathematics ,232 , 252 – 261.

Liu, C., Gong, Z., Shen, B., & Feng, E. (2013). Modelling and optimal controlfor a fed-batch fermentation process. Applied Mathematical Modelling , 37 ,695 – 706.

Logist, F., Houska, B., Diehl, M., & Van Impe, J. (2010). Fast pareto setgeneration for nonlinear optimal control problems with multiple objectives.Structural and Multidisciplinary Optimization, 42 , 591–603.

29

Logist, F., & Van Impe, J. (2012a). Multi-objective dynamic optimisation ofcyclic chemical reactors with distributed parameters. Chemical Engineer-ing Science, 80 , 429 – 434.

Logist, F., & Van Impe, J. (2012b). Novel insights for multi-objective optimi-sation in engineering using normal boundary intersection and (enhanced)normalised normal constraint. Structural and Multidisciplinary Optimiza-tion, 45 , 417–431.

Lopez-Negrete, R., Patwardhan, S. C., & Biegler, L. T. (2009). Approxi-mation of arrival cost in moving horizon estimation using a constrainedparticle filter. In R. M. de Brito Alves, C. A. O. do Nascimento, & E. C.Biscaia (Eds.), 10th International Symposium on Process Systems Engi-neering: Part A (pp. 1299 – 1304). Elsevier volume 27.

Lucia, S., Finkler, T., & Engell, S. (2013). Multistage nonlinear model pre-dictive control applied to semi-batch polymerization reactor under uncer-tainity. Journal of Process Control , 23 , 1306 – 1319.

Lucia, S., Tatulea-Codrean, A., Schoppmeyer, C., & Engell, S. (2014). Anenvironment for the efficient testing and implementation of robust NMPC.In 2014 IEEE Conference on Control Applications (CCA) (pp. 1843–1848).

Masehian, E., & Sedighizadeh, D. (2010). A multi-objective PSO-based al-gorithm for robot path planning. In 2010 IEEE International Conferenceon Industrial Technology (ICIT) (pp. 465–470).

Messac, A., Ismail-Yahaya, A., & Mattson, C. (2003). The normalized nor-mal constraint method for generating the Pareto frontier. Structural &Multidisciplinary Optimization, 25 , 86–98.

Miettinen, K. (1998). Nonlinear Multiobjective Optimization. Springer US.

Montoya, F. G., Manzano-Agugliaro, F., Lpez-Mrquez, S., Hernndez-Escobedo, Q., & Gil, C. (2014). Wind turbine selection for wind farmlayout using multi-objective evolutionary algorithms. Expert Systems withApplications , 41 , 6585 – 6595.

Nagy, Z. (2008). OptCon - An efficient tool for rapid prototyping of nonlinearmodel predictive control applications. In Proceedings of AIChE AnnualMeeting 2008 (pp. 16–21).

30

Nagy, Z., Findeisen, R., Diehl, M., Allgower, F., & Bock, H. G. (2000). Realtime feasibility of nonlinear predictive control for large scale processes - acase study. In Proceedings of American Control Conference (pp. 4249 –4253). IEEE volume 6.

Naidu, D. S., Fernando, T., & Fister, K. R. (2011). Optimal control indiabetes. Optimal Control Applications and Methods , 32 , 181–184.

Nimmegeers, P., Telen, D., Logist, F., & Van Impe, J. (2016). Multi-objectiveoptimization of dynamic biological networks under parametric uncertainty.BMC Systems Biology , Submitted .

Pontryagin, L. S., Boltyanskii, V. G., Gamkrelidze, R. V., & Mishechenko,E. F. (1962). The Mathematical Theory of Optimal Processes . WILEYInterscience.

Qu, C. C., & Hahn, J. (2009). Computation of arrival cost for moving horizonestimation via unscented Kalman filtering. Journal of Process Control , 19 ,358 – 363.

Quirynen, R., Gros, S., & Diehl, M. (2013). Fast auto generated acadointegrators and application to mhe with multi-rate measurements. In 2013European Control Conference (ECC) (pp. 3077–3082).

Rao, C., Rawlings, J., & Mayne, D. (2003). Constrained state estimation fornonlinear discrete-time systems: stability and moving horizon approxima-tions. Automatic Control, IEEE Transactions on, 48 , 246–258.

Rao, C. V., & Rawlings, J. B. (2002). Constrained process monitoring:Moving-horizon approach. AIChE Journal , 48 , 97–109.

Rawlings, J. B. (2000). Tutorial overview of model predictive control. IEEEControl Systems Magazine, 20:3 , 38 – 52.

Ricker, N., & Lee, J. (1995). Nonlinear modeling and state estimation forthe Tennessee Eastman challenge process. Computers & Chemical Engi-neering , 19 , 983 – 1005.

Robertson, D. G., Lee, J. H., & Rawlings, J. B. (1996). A moving horizon-based approach for least-squares estimation. AIChE Journal , 42 , 2209–2224.

31

Romanenko, A., & Santos, L. (2007). A nonlinear model predictive controlframework as free software: Outlook and progress report. In Assessmentand Future Directions of Nonlinear Model Predictive Control (pp. 229–238). Springer Berlin Heidelberg volume 358 of Lecture Notes in Controland Information Sciences .

Ross, I. M., & Karpenko, M. (2012). A review of pseudospectral optimalcontrol: From theory to flight. Annual Reviews in Control , 36 , 182 – 197.

Saerens, B., Vandersteen, J., Persoons, T., Swevers, J., Diehl, M., & Van denBulck, E. (2009). Minimization of the fuel consumption of a gasoline engineusing dynamic optimization. Applied Energy , 86 , 1582 – 1588.

Sanchis, J., Martinez, M., Blasco, X., & Salcedo, J. (2008). A new perspectiveon multiobjective optimization by enhanced normalized normal constraintmethod. Structural and Multidisciplinary Optimization, 36 , 537–546.

Schiesser, W. E. (1991). The Numerical Method of Lines . San Diego, CA,USA: Academic Press.

Sendın, J. H., Alonso, A. A., & Banga, J. R. (2009). 2nd InternationalWorkshop on Practical Applications of Computational Biology and Bioin-formatics. chapter Multi-Objective Optimization of Biological Networksfor Prediction of Intracellular Fluxes. (pp. 197–205). Berlin, Heidelberg:Springer Berlin Heidelberg.

Srinivasan, B., Palanki, S., & Bonvin, D. (2003). Dynamic optimization ofbatch processes I: Characterization of the nominal solution. Computers &Chemical Engineering , 27 , 759–760.

Stamati, I., Logist, F., Van Derlinden, E., Gauchi, J.-P., & Van Impe, J.(2014). Optimal experimental design for discriminating between micro-bial growth models as function of suboptimal temperature. MathematicalBiosciences , 250 , 69 – 80.

Telen, D., Vallerio, M., Cabianca, L., Houska, B., Van Impe, J., & Logist,F. (2015). Approximate robust optimization of nonlinear systems underparametric uncertainty and process noise. Journal of Process Control , 33 ,140 – 154.

32

Telen, D., Vercammen, D., Logist, F., & Van Impe, J. (2014). Robustifyingoptimal experiment design for nonlinear, dynamic (bio)chemical systems.Computers & Chemical Engineering , 71 , 415 – 425.

TOMLAB Optimization (2011). PROPT v7.8. http://www.tomdyn.com/.

Ungarala, S. (2009). Computing arrival cost parameters in moving horizonestimation using sampling based filters. Journal of Process Control , 19 ,1576 – 1588.

Vallerio, M., Hufkens, J., Van Impe, J., & Logist, F. (2015). An interactivedecision-support system for multi-objective optimization of nonlinear dy-namic processes with uncertainty. Expert Systems with Applications , 42 ,7710 – 7731.

Vallerio, M., Telen, D., Cabianca, L., Manenti, F., Van Impe, J., & Logist, F.(2016). Robust multi-objective dynamic optimization of chemical processesusing the sigma point method. Chemical Engineering Science, 140 , 201 –216.

Vercammen, D., Logist, F., & Van Impe, J. (2014). Dynamic estimation ofspecific fluxes in metabolic networks using non-linear dynamic optimiza-tion. BMC Systems Biology , 8 , 1–22.

Vercammen, D., Logist, F., & Van Impe, J. (2016). Online moving horizonestimation of fluxes in metabolic reaction networks. Journal of ProcessControl , 37 , 1 – 20.

Verhelst, C., Logist, F., Van Impe, J., & Helsen, L. (2012). Study of the op-timal control problem formulation for modulating air-to-water heat pumpsconnected to a residential floor heating system. Energy and Buildings , 45 ,43 – 53.

Vukov, M., Domahidi, A., Ferreau, H. J., Morari, M., & Diehl, M. (2013).Auto-generated algorithms for nonlinear model predictive control on longand on short horizons. In 2013 IEEE 52nd Annual Conference on Decisionand Control (CDC) (pp. 5113–5118).

Vukov, M., Gros, S., Horn, G., Frison, G., Geebelen, K., Jørgensen, J.,Swevers, J., & Diehl, M. (2015). Real-time nonlinear MPC and MHE for

33

a large-scale mechatronic application. Control Engineering Practice, 45 ,64 – 78.

Wachter, A., & Biegler, L. (2006). On the implementation of a primal-dualinterior point filter line search algorithm for large-scale nonlinear program-ming. Mathematical Programming , 106 , 25–57.

Zavala, V. M., & Biegler, L. T. (2009a). The advanced-step NMPC controller:Optimality, stability and robustness. Automatica, 45 , 86 – 93.

Zavala, V. M., & Biegler, L. T. (2009b). Optimization based strategies foroperation of low-density polyethylene tubular reactors: nonlinear modelpredictive control. Computers & Chemical Engineering , 33 , 1735 – 1746.

Zavala, V. M., & Biegler, L. T. (2009c). Optimization-based strategies forthe operation of low-density polyethylene tubular reactors Moving horizonestimation. Computers & Chemical Engineering , 33 , 379 – 390.

34