Structural and Multidisciplinary Optimization manuscript No.(will be inserted by the editor)

Differential Geometry Tools for Multidisciplinary DesignOptimization, Part I: Theory

Craig Bakker · Geoffrey T. Parks

Received: date / Accepted: date

Abstract Analysis within the field of MultidisciplinaryDesign Optimization (MDO) generally falls under theheadings of architecture proofs and sensitivity infor-mation manipulation. We propose a differential geom-etry (DG) framework for further analyzing MDO sys-tems, and here, we outline the theory undergirding thatframework: general DG, Riemannian geometry for usein MDO, and the translation of MDO into the languageof DG. Calculating the necessary quantities requiresonly basic sensitivity information (typically from thestate equations) and the use of the implicit function the-orem. The presence of extra or non-differentiable con-straints may limit the use of the framework, however.Ultimately, the language and concepts of DG give usnew tools for understanding, evaluating, and develop-ing MDO methods; in Part I, we discuss the use of thesetools and in Part II, we provide a specific application.

Keywords Multidisciplinary Design Optimization ·Differential Geometry · Design Analysis

1 Introduction

Multidisciplinary Design Optimization (MDO) is con-cerned with the optimization of systems that have cou-pled subsystems or disciplines. These systems typically

Previously presented as On the Application of DifferentialGeometry to MDO. In: 12th Aviation Technology, Integrationand Operations (ATIO) Conference and 14th AIAA/ISSMOMultidisciplinary Analysis and Optimization Conference,AIAA, Indianapolis, Indiana

C. Bakker · G.T. ParksDepartment of EngineeringUniversity of CambridgeCambridge, United KingdomCB2 1PZ E-mail: ckrb2@cam.ac.uk

have several disciplines, each with design variables spe-cific to that discipline; state variables that representthe outputs of their respective disciplines; and designvariables common to more than one discipline – localdesign, state, and global design variables, respectively.MDO applies various decomposition strategies, knownas architectures, to these systems to make them moreamenable to optimization. These strategies include An-alytical Target Cascading (ATC) (Kim 2001), Simul-taneous Analysis and Design (SAND) (Haftka 1985),Quasiseparable Decomposition (QSD) (Haftka andWat-son 2005), Multidisciplinary Feasible (MDF) (Crameret al. 1994), Collaborative Optimization (CO) (Braunand Kroo 1997), Bi-Level Integrated System Synthesis(BLISS) (Sobieszczanski-Sobieski et al. 1998), Concur-rent Subspace Optimization (CSSO) (Bloebaum et al.1992), and Individual Discipline Feasible (IDF) (Crameret al. 1994); naming variations exist within the litera-ture, but these are the most commonly used appella-tions. Theoretical MDO work has focused on architec-ture equivalence or convergence proofs and the determi-nation of design sensitivities for MDO, but few architec-tures have these proofs; most decomposition methodsthat have such proofs require special problem structuresnot typically found in MDO (Haftka and Watson 2006).In this paper, we develop a differential geometry (DG)framework for facilitating the analysis and evaluationof MDO methods: Part I details the underlying theory,and Part II uses that theory to investigate QSD.

2 Review of MDO analysis

In light of the extensive analytical work we are about toundertake, beginning with this paper, it is appropriateto review the analysis which has previously been done

2 Craig Bakker, Geo!rey T. Parks

within the field. This work falls broadly under two cat-egories: architecture proofs and the manipulation andanalysis of sensitivity information.

2.1 Architecture convergence and equivalence

There are not many convergence or equivalence proofsfor the various architectures in the MDO literature.Convergence is a fairly self-explanatory concept, butit should be noted that, in the context of MDO, equiv-alence refers to the correspondence between (optimal)solutions of the decomposed problem and (optimal) so-lutions of the original problem.

Most of the extant MDO analysis has been focusedaround ATC. ATC was developed by Kim (2001), andhe conjectured that ATC was convergent based on itssimilarity to Hierarchical Overlapping Coordination, adecomposition-based optimization method with provenconvergence. Michelena et al. (2003) later proved thisconjecture correct given a hierarchical problem struc-ture and certain coordination schemes.

Considerable effort has also been put into the in-vestigation of penalty functions and their weightingswithin an ATC context, as these can strongly affectconvergence performance and efficiency. Han and Pa-palambros (2010) proved the existence of convergentweights with L! norms (though not how to find suchweights); however, it seems that most of the literature inthis area has focused on Augmented Lagrangian coor-dination (Kim et al. 2006; Tosserams et al. 2006, 2009).

Work in solution equivalence has been done withQSD. With convexity assumptions and the quasisep-arable problem structure, Haftka and Watson (2005)proved the equivalence of optimal solutions (both globaland local) between the original and decomposed prob-lems while also showing that feasible points in one prob-lem corresponded to feasible points in the other.

Conversely, convergence issues for CO in both the-ory and practice have been highlighted; the problemarises because the penalty functions used lead to eithernondifferentiability or vanishing Lagrange multipliersat optimal points (Alexandrov and Lewis 2002). Somesolutions have been proposed to keep the concept un-derlying the CO architecture while overcoming these is-sues: Modified Collaborative Optimization (MCO) fromDeMiguel and Murray (2000) and Enhanced Collabora-tive Optimization (ECO) from Roth and Kroo (2008).DeMiguel and Murray (2001; 2006) provided equiva-lence and convergence proofs for their proposal, but itis interesting to note how similar their formulation endsup being to that of ATC. This is meant not as a dispar-agement of their work but rather as an observation that

certain solution methods (i.e. a combination of objec-tive and penalty functions at each level of a multi-levelhierarchical optimization problem) seem to lend them-selves more easily to convergence proofs than others.

2.2 Sensitivity calculations

Some work has been done to translate sensitivity infor-mation between architectures (Alexandrov and Lewis2000, 2003), but so far, this has only been done betweenthe monolithic architectures (MDF, IDF, and SAND).Decomposition in the distributed architectures seemsto be a significant obstacle to such translation.

Using the implicit function theorem, Sobieszczanski-Sobieski (1990a) developed the Global Sensitivity Equa-tions (GSE) to calculate cross-disciplinary sensitivitieswithout needing costly multidisciplinary analyses; hethen extended the method for the calculation of higher-order sensitivities (Sobieszczanski-Sobieski 1990b). Al-though this did not completely eliminate the costs in-volved in sensitivity analysis, it did make substantialreductions possible: for architectures that require thisinformation for their decomposition procedures (e.g.BLISS and CSSO), a mechanism like the GSE is neces-sary for good performance on coupled systems.

Optimum Sensitivity Analysis (OSA) is used to cal-culate the derivatives of an optimal solution with re-spect to problem parameters (i.e. quantities that wereheld constant during the optimization) (Barthelemy andSobieszczanski-Sobieski 1983a,b). These calculations en-able perturbation analyses to be done on the solutionwithout needing costly re-optimizations each time. Thisis particularly useful for distributed or multi-level MDOarchitectures because OSA can then be used to obtainderivatives cheaply from the lower level problem for usein the upper level problem (or vice versa).

All of this is mainly concerned with the use of basicsensitivity information to gain further knowledge aboutthe system – it does not consider how sensitivity infor-mation is calculated in the first place. The calculationof that basic information is an important area of re-search in its own right. See van Keulen et al. (2005) fora review thereof. Martins and Hwang (2012) also pro-vide a review of the different methods available for cal-culating derivatives in multidisciplinary systems; theyinclude methods for obtaining basic sensitivity informa-tion and for obtaining cross-disciplinary sensitivities.

3 Differential geometry theory

The basic idea behind our framework is that MDO con-sists of optimization done on manifolds – which are es-

Di!erential Geometry Tools for Multidisciplinary Design Optimization, Part I: Theory 3

sentially higher-dimensional versions of surfaces – andthat the language of DG can thus be used to describethe relevant processes and quantities; the constrainedoptimizations in Euclidean spaces are reconceptualizedas unconstrained optimizations on Riemannian mani-folds embedded in those Euclidean spaces. We do thistranslation in Section 4, but first, we give the theory un-derlying that translation. The explanation given in thissection is not meant to be exhaustive – it merely givesthe context and background necessary to make laterderivations comprehensible. More detailed treatmentsare available elsewhere. The material for this introduc-tion is mainly drawn from Boothby (1986) and othersare cited where pertinent.

3.1 Basic concepts in differential geometry

Differential geometry is concerned with doing mathe-matics (such as calculus) on generally non-Euclideanspaces. In Euclidean geometry, the basic structures ofspace are linear: lines, planes, and their analogues inhigher dimensions; DG, on the other hand, deals withmanifolds (which may be Euclidean but often are not).Manifolds are abstract mathematical spaces which lo-cally resemble the spaces described by Euclidean geom-etry but may have a more complicated global structure.They can also be considered extrinsically or intrinsi-cally: an extrinsic view of a manifold considers it as be-ing embedded in a higher-dimensional Euclidean spacewhereas the intrinsic view considers the manifold moreabstractly without any need for a surrounding space(Ivancevic and Ivancevic 2007).

In the context of our reconceptualization of MDOas discussed earlier, this means that we could considerthe constraint manifolds extrinsically, as being embed-ded in the total design space (which would be a higher-dimensional Euclidean space), or intrinsically, withoutreference to the total design space in which it is em-bedded. The extrinsic view would see the manifold asa set of points in !n, but, as we will show in thissection, manifolds have additional mathematical struc-tures which do not apply to sets of points in general.

Consider a manifold M of dimension n (denoted asan n-manifold). About any point p " M , there existsa coordinate neighbourhood, also known as a chart,consisting of a neighbourhood U of p and a mapping! : U # U , where U is an open subset of !n. If M is amanifold with a boundary "M , and p " "M , then

U $ Hn, Hn =!"

x1, x2, . . . , xn#" !n|xn % 0

$(1)

Note that "M is itself a manifold of dimension n&1.The mapping ! need not be unique, however; Fig. 1

shows an example of these charts. This mapping is whatallows us to take the familiar analyses done in (flat)Euclidean space and apply them to (curved) manifolds.Ultimately, the chart is what makes it possible to definethings like coordinate reference frames and derivatives.

Fig. 1 Overlapping charts on a 2-manifold in 3-space(Boothby 1986, used with permission)

Differentiable manifolds are simply manifolds suchthat ! is differentiable at every p " M . For the purposesof this discussion, we assume that all the manifolds weencounter are sufficiently differentiable that questionsof discontinuities or singularities do not arise.

Given any two manifolds M and N (and !n is amanifold, albeit a globally Euclidean one), it is possi-ble to construct a product manifold M 'N with struc-ture inherited from M and N . Conversely, a subman-ifold N of a manifold M is a subset of points in Msuch that N itself is also a manifold, and submani-folds inherit certain properties from the manifolds inwhich they are embedded. Identifying product mani-fold or submanifold structures is valuable because itcan simplify calculations done on such manifolds andhelp to show the relationships between different mani-folds. Figure 2 gives an example of some submanifolds:the black curve is a submanifold of the surface in whichit is embedded, which is in turn a submanifold of thesurrounding Euclidean space !3. For more on productmanifolds, submanifolds, and the properties of both, seeBoothby (1986) or Yano and Kon (1984).

The tangent space at p is the vector space consistingof all vectors tangent to M at p, and it is denoted byTp (M). It is itself an n-manifold. The tangent bundleT (M) consists of the union of the tangent spaces at

4 Craig Bakker, Geo!rey T. Parks

Fig. 2 A 1-D submanifold of a 2-D manifold in !3

each point p " M , and it is a 2n-manifold. Vector fieldson the manifold “live” in the tangent bundle. The tan-gent spaces and the tangent bundle also have duals – thecotangent spaces T "

p (M) and cotangent bundle T " (M),respectively, where covector fields “live” (Ivancevic andIvancevic 2007); the dual of a vector space is the spaceof linear functionals such that the product of a memberof a vector space with a member from the dual space re-turns a scalar. These tangent and cotangent spaces areimportant for defining higher-order tensors, too, andthey play a role in derivative calculations as well as thedetermination of angles and lengths on the manifold.

Let F : M # N be a smooth mapping betweenmanifolds. Given F , which maps between points of thetwo manifolds, we can induce transformations of othermathematical objects between the two manifolds, de-noting such inductions with a correctly placed (. Forexample, F" : Tp (M) # TF (p) (N). A subscript ( indi-cates that the induced mapping goes in the same “di-rection” as the original mapping, whereas a superscriptindicates that the induced mapping goes in the oppo-site “direction” as the original mapping; see ChapterFour of Boothby (1986) for further explanation.

This idea of induced mappings allows us to developcoordinate bases for the tangent and cotangent spaces,using the mapping !, at p. Consider !" : Tp (M) #T!(p) (!n). In keeping with typical DG notation, wedefine the basis vectors of T!(p) (!n) as

"

"xi, i = 1, 2, . . . , n (2)

We can then define the basis vectors of Tp (M) as

Ei = !#1"

%"

"xi

&, i = 1, 2, . . . , n (3)

The Ei’s thus form a basis for Tp (M), and an anal-ogous procedure can be performed for the cotangentbasis vectors (represented as dxi in T "

!(p) (!n) and Ei

in T "p (M)). Figure 3 shows how the grid in Euclidean

space (with its associated basis vectors in the direc-tions of the coordinate axes) are mapped back to themanifold with the basis vectors correspondingly trans-formed. This shows the connection, mentioned earlier,between the manifold’s chart and its coordinates.

Fig. 3 Grid with basis vectors on a manifold (Boothby 1986,used with permission)

Moreover, although there are an infinite number ofpossible bases that could be chosen for each space, foreach set of basis vectors Ei, there exists a unique setof dual basis vectors Ei such that Ei (Ej) = #ij , where#ij is the Kronecker delta, and the basis vectors as justdefined form such a pairing. Keep in mind, however,that since !, in general, varies across the manifold, thetangent and cotangent basis vectors will also vary frompoint to point. Figure 4 shows a set of orthonormalbasis vectors varying over the manifold.

Fig. 4 A field of basis vectors on a manifold (Boothby 1986,used with permission)

Di!erential Geometry Tools for Multidisciplinary Design Optimization, Part I: Theory 5

We will use the notation

Ei ="

"wi, Ei = dwi (4)

The significance of the subscript and superscript in-dices is explained in Section 3.2. Given this notation, avector X on the manifold can be expressed as

X = X1 "

"w1+X2 "

"w2+ . . .+Xn "

"wn(5)

and a covector $ would be expressed as

$ = $1dw1 + $2dw

2 + . . .+ $ndwn (6)

Consider now a scalar function f . Its differential dfand directional derivative in the direction X are given,respectively, by

df ='

i

"f

"widwi (7)

Xf ='

i

Xi "f

"wi(8)

3.2 Tensors and tensor notation

A tensor is a geometrical object that behaves a certainway under coordinate transformations; a tensor repre-sentation independent of the chosen basis is unneces-sarily abstract for our purposes, so we will use indexnotation. Tensors come in three basic types: contravari-ant, covariant, and mixed. These types are denoted byindices: contravariant indices are superscripted, covari-ant indices are subscripted, and the order of the tensoris determined by the number of indices. For example, afirst-order contravariant tensor (i.e. a vector) would bedenoted by vi, and a first-order covariant tensor (i.e. acovector) would be vi. It is understood that the tensorhas a distinct value for each i = 1, 2, . . . , n, where n isthe dimension of the space under consideration. Con-tinuing on, tij , tij , and tij would be second-order con-travariant, covariant, and mixed tensors, respectively.The tensor Ri

jkl would be a fourth-order mixed tensorwith one contravariant index and three covariant in-dices (Ivancevic and Ivancevic 2007). To link back tothe discussion of tangent and cotangent spaces, the co-variant indices of tensors “live” in the tangent space andproducts thereof whereas the contravariant indices livein the cotangent space and products thereof. This mat-ters because covariant and contravariant indices behavedifferently under covariant differentiation (as shown in

Section 3.3) and coordinate transformations. We alsomention the Einstein summation convention. The con-vention is that repeated indices are summed over:

aibi ='

i

aibi (9)

Index notation with the summation convention willbe used throughout this paper unless otherwise noted.

3.3 Riemannian geometry

A Riemannian manifold is a differentiable manifold witha symmetric, positive definite bilinear form (known asthe metric tensor). The metric tensor is an importanttool for doing calculations on the manifold. Given a Rie-mannian manifold M , the metric tensor gij defines aninner product on Tp (M) and this makes it possible toperform a number of different mathematical operationson the manifold. In general, gij is defined by

gij = Ei ·Ej (10)

where Ei and Ej are as previously defined. For example,for Cartesian coordinates in !n, this is just

gij = Ei ·Ej = #ij (11)

where #ij is the Kronecker delta. The inner product fora pair of vectors is then defined as

)u,v* = gijuivj (12)

Lengths on the manifold are calculated with gij :

ds2 = gijdwidwj (13)

The metric tensor is one of the properties inheritedby submanifolds and product manifolds – the subman-ifold and product manifold structures are reflected inthe forms of their respective metric tensors. Christoffelsymbols and curvature tensors can also be derived fromgij . If gij is known at a point on M , it is possible tocalculate the Christoffel symbols and the curvature ofthe manifold at that point:

% ikl =

1

2gim

%"gmk

"wl+

"gml

"wk& "gkl

"wm

&(14)

Rijkl =

"% ijl

"wk&

"% ijk

"wl+ %m

jl %imk & %m

jk%iml (15)

Rij = Rlilj , R = gijRij (16)

6 Craig Bakker, Geo!rey T. Parks

where % ikl is a Christoffel symbol, Ri

jkl is the Riemanncurvature tensor, Rij is the Ricci curvature tensor, Ris the scalar curvature, gij is the inverse of gij , n is the

manifold dimension, and w =!w1, . . . , wn

$Tare the

domain variables (i.e. the manifold coordinates). Wemention also the Weyl curvature tensor, Cijkl; thougha relevant quantity, its formula is too long to show con-veniently here. If gij exists, then it is positive definite,and thus its inverse also exists (and is positive definite).The Christoffel symbols measure how the basis vectorschange along the manifold

+EiEj = % kijEk (17)

and they show up in two important places: the covariantderivative and the geodesic equation; % i

jk are essentiallyintermediate quantities – they are typically used forcalculating other pieces of information. The geodesicequation is

wi + % ijkw

jwk = 0 (18)

Solutions of the geodesic equation are the paths thatparticles moving along the manifold would take if theywere not subjected to any external forces (Ivancevicand Ivancevic 2007). They are to curved spaces whatstraight lines are to flat spaces: great circles on a sphereare a familiar example. Minimal geodesics are also usedto define distances between points on a manifold.

The covariant derivative, denoted with a subscriptsemi-colon, does two things: it projects the derivativeonto the tangent space (Boothby 1986), and it main-tains the tensorial character of whatever it derivates(Ivancevic and Ivancevic 2007). The first few formulaefor the covariant derivative are

vi;k = vi,k + % ijkv

j (19)

vi;k = vi,k & % jikvj (20)

tikl;q = tikl,q + % iqst

skl & % s

kqtisl & % s

lqtiks (21)

where a subscript comma denotes an ordinary partialderivative (i.e. vi,j = "vi

"wj ). Notice how covariant andcontravariant indices are differentiated differently; also,gij;k = 0 (Szekeres 2004). In the rest of this paper,where total derivatives need to be distinguished fromregular partial derivatives, d will be used instead of ",but " will generally be used to indicate the multivariatenature of the derivatives being taken.

The Riemann curvature tensor is somewhat moredifficult to interpret: it measures the “acceleration” be-tween geodesics (Szekeres 2004); Rij describes how a

neighborhood, as the points in the neighborhood movealong geodesics, changes in volume, and Cijkl describeshow that neighborhood changes in shape. These twotensors capture the information contained in Ri

jkl (Pen-rose 1989). Each of these curvature tensors shows, in itsown way, the nature and extent to which the manifolddeviates from flat space.

4 Translating MDO into differential geometry

With the requisite background in DG, we can now trans-late the mathematical form of an MDO problem intothe relevant geometric terminology; we will use a gen-eral MDO formulation so as to make our translation,and thus work done with the framework, similarly gen-eral and widely applicable within MDO.

4.1 Mathematical definition of MDO

Consider a generic MDO problem as

min f(x,y, z) (22)

g(x,y, z) , 0 (23)

h(x,y, z) = 0 (24)

where x is the vector of local design variables, y is thevector of state variables, z is the vector of global designvariables, g is the vector of inequality constraints, andh is the vector equation that defines the state variables;(24) is just a rearrangement of the state equations

yi = &i(x(i), y(i), z), i = 1, 2, . . . (25)

where x(i) is the set of local variables for discipline i andy(i) is the set of all state variables excluding yi. For thepurposes of this paper, we will only focus on the equal-ity constraints. We recognize, though, that the inequal-ity constraints will eventually have to be considered (seeSection 6.1). The variables can be further simplified bylumping them together: w = {x z} and v = {w y}

Defining the MDO problem this way raises somequestions. Firstly, there is the question of notation:there are several different notations in the MDO lit-erature (Cramer et al. 1994; Martins and Lambe 2013).This particular notation was used for the clarity andsimplicity which it would lend to our later derivations.Secondly, there is the question of terminology: whatwe have labeled as state variables have been variouslydescribed as state, coupling, or linking variables, eachwith slightly different roles in the overall optimization.

Di!erential Geometry Tools for Multidisciplinary Design Optimization, Part I: Theory 7

For the purposes of this work, we consider state vari-ables to be outputs of their respective disciplines; thesevariables would then, typically, be inputs to other dis-ciplines, but they could be used only within other stateequations belonging to their discipline. The key point isthat state variables are not seen as directly controllableby the designer: lift and drag, for example, would bestate variables from an aerodynamics discipline in air-craft design. This has been done to mirror real-worlddesign problems and for ease of derivation later on.

Manifolds are useful for MDO because the stateequations, which describe the interactions between thedifferent disciplines and state variables, implicitly de-fine a manifold. This is the feasible design manifold –the space of all feasible designs. Let Mfeas be the man-ifold defined by (24).

4.2 Tangent vectors

Fig. 5 A 2-manifold in 3-space with tangent vectors

Consider a 2-manifold embedded in 3-space as inFig. 5. The tangent vectors t1 and t2 can be given as

t1 =

()

*

10"y"x1

+,

- , t2 =

()

*

01"y"x2

+,

- (26)

It is possible to generalize these formulae to a mani-fold of arbitrary dimension and co-dimension. Considerthe manifold defined by (24) and assume that the mani-fold is an n-dimensional manifold embedded in m-space(co-dimension m& n). The tangent vectors are then

ti ="v

"wi=

."v1

"wi, . . . ,

"vm

"wi

/T

(27)

=

.0, . . . , 0, 1, 0, . . . , 0,

"y1

"wi, . . . ,

"ym#n

"wi

/T

for i = 1, 2, . . . , n where the 1 is in the i-th slot andthe necessary derivatives are calculated from (24) usingthe implicit function theorem (with y as an implicitfunction of w).

It is straightforward to show that the tangent vec-tors are all linearly independent (though not all mutu-ally orthogonal or of unit length). As such, the ti’s forma basis for the tangent space of the manifold, and so wecan define them as the basis vectors that will be usedthroughout the rest of the paper: Ei = ti =

""wi .

It is interesting to note some characteristics of thischoice of tangent vectors. Each tangent vector corre-sponds directly to a design variable. The disciplinarysubspaces of the tangent space are therefore just the lin-ear combinations of the tangent vectors correspondingto the discipline’s design variables. This also simplifiesthe derivation of later geometric quantities in terms ofthe pertinent partial derivatives. Defining the tangentvectors gives us our coordinate reference frame and thusour metric tensor (which is critical for future quantita-tive analysis). We do not use this information in PartII, but we highlight its use elsewhere in Section 5.

4.3 Derivation of geometric quantities

Assuming sufficient differentiability, and given the def-inition of (10), gij is defined by

gij ="vk

"wi

"vk

"wj= #ij +

"yk

"wi

"yk

"wj(28)

The same results could be obtained by using sub-manifold theory and the metric induced from the em-bedding space (!m), but this way is more geometricallyintuitive. The Christoffel symbols are then

% ikl = gim

"ys

"wm

"2ys

"wk"wl(29)

and the derivatives of % ijk are

"% ikl

"wj=

"gim

"wj

"ys

"wm

"2ys

"wk"wl

+ gim"2ys

"wm"wj

"2ys

"wk"wl+ gim

"ys

"wm

"3ys

"wk"wl"wj(30)

where

8 Craig Bakker, Geo!rey T. Parks

"gim

"wj= &gin

%"2ys

"wj"wn

"ys

"wp+

"ys

"wn

"2ys

"wj"wp

&gpm

(31)

Finally, Rijkl, Rij , and R can be calculated from

(15) and (16); the explicit formulae have been omitteddue to their length. Fortunately for the sake of prac-tical calculation, the third implicit derivatives in % i

kl,j

cancel out in the determination of the Riemann curva-ture tensor, so it and the other curvature tensors onlydepend on the second implicit derivatives.

The particular structure of MDO problems thus sim-plifies the calculations so that the relevant quantitiescan be computed from the implicit derivatives of thestate variables with respect to the design variables. Im-plicit derivatives can be calculated from the basic sensi-tivity information using the implicit function theorem.

4.4 The objective function and its derivatives

It is now necessary to see how the original MDO objec-tive function is affected by this translation. The sim-plest way forward is to define the objective function' (w) = f (w,y (w)) and proceed from there. For ascalar function, the covariant derivative is identical tothe regular derivative:

';i = ',i ="'

"wi=

"f

"wi+

"f

"yk"yk

"wi(32)

This is just the reduced gradient of f . However, tak-ing the covariant derivative again does not result in thereduced Hessian. The reduced Hessian and the secondcovariant derivative of ' are, respectively,

',ij ="2f

"wi"wj+

"2f

"wi"yk"yk

"wj+

"2f

"wj"yk"yk

"wi

+"2f

"ym"yk"yk

"wi

"ym

"wj+

"f

"yk"2yk

"wi"wj(33)

';ij = ',ij & % lij',l (34)

This should not affect the optimality conditions, as',l = 0 at an optimum point. Calculating the objectivefunction derivatives like this demonstrates the use ofthe covariant derivative, which we introduced in Sec-tion 3.3, and we use these objective function covariantderivatives in research described in Section 5.

5 Use of the differential geometry frameworkelsewhere

We give here an overview of how the framework hasbeen, is being, and may be utilized. In Sections 3 and4, we provided more theory than is necessary for our ap-plication in Part II, and we did this with the intentionthat the theory outlined here be the foundation for thefuture evaluation and development of MDO methods.Our goal is to develop a framework for a wide range ofMDO analysis. As such, we wish to outline the otherwork related to this theory, mention which parts of ourframework that research is drawing from, and show howthat work interacts with identified needs within MDO.The results detailed in Part II provide a concrete exam-ple, valuable in its own right, of the kind of work thisframework can enable. Part II, with the rest of Section5, serves as incentive for the effort of putting togetherSections 3 and 4.

5.1 Foundational work

Certain pieces of work laid the foundation and providedthe motivation for our framework. A notable item wasthe development of a coupling metric for MDO prob-lems (Bakker et al. 2013a). This coupling metric ad-dresses the desire expressed by Agte et al. (2010) fora unified coupling measure; our metric measures bothcoupling bandwidth and coupling strength but can bereduced to a single scalar value. We have also shownthat our coupling measure extends to Multi-ObjectiveOptimization (MOO) problems. This is a step towardsconnecting MDO and MOO as desired by Tolson andSobieszczanski-Sobieski (1985) and Lewis and Mistree(1998). Our coupling metric was developed using gij .

Secondly, we used our framework to give a geomet-ric interpretation of several common MDO architec-tures (Bakker et al. 2012). From this, we made somequalitative observations about architecture behaviour –a step towards the analysis desired by Sobieszczanski-Sobieski and Haftka (1997) while being valuable for ourown work. Having defined the architecture manifolds,we need only to substitute them into our future results.In other words, the analysis we do with our frameworkcan be generalized to those architectures. Our work hereused some submanifold concepts as well as gij .

Thirdly, we have analytically linked optimization al-gorithm behaviour in MDO to manifold properties bycombining theory from Ordinary Differential Equationswith differential geometry (Bakker et al. 2013b). Weconnected algorithm stability with manifold curvaturewhich in turn relates to our coupling metric. This can

Di!erential Geometry Tools for Multidisciplinary Design Optimization, Part I: Theory 9

show how different architectures (and thus the architec-tures’ manifolds) affect algorithm stability on a givenproblem. Our work here addresses both the interest inconvergence expressed by Sobieszczanski-Sobieski andHaftka (1997) and the desire of Agte et al. (2010) to usecoupling information to guide decomposition decisions.These results drew on covariant derivatives (includingbut not limited to the objective function), the geodesicequation, and the curvature tensors.

5.2 Current applications and research directions

Following up on work already accomplished, we arecurrently working on using our coupling metric to de-velop coupling suspension techniques for MDO prob-lems and to further investigate architecture characteris-tics and performance. We are also continuing to pursuethe link we have established between MDO and MOO:we can combine our coupling metric with current MOOtechniques and numerical solution methods for PartialDifferential Equations to generate smart Pareto fronts(Bakker and Parks 2014); a smart Pareto front has solu-tion points concentrated in areas of high interest (Matt-son et al. 2004). This further serves our intention ofmaking connections between MOO and MDO. Similarmethods may also be used improve data sampling tech-niques in metamodelling. All of this uses gij .

In a new area, we are looking at applying Rieman-nian optimization algorithms to MDO problems; themathematical structure provided by our framework nowmakes it possible to implement these algorithms com-putationally. Given that they have been derived for op-timization on curved spaces, they may be more effectivethan their Euclidean counterparts on MDO manifolds.These Riemannian algorithms require using covariantdifferentiation (and thus % i

jk).

5.3 Areas for future research

Researchers have identified three other related areas ofoptimization which they would also like to incorporateinto MDO: optimal control (Allison and Herber 2013;Giesing and Barthelemy 1998), uncertainty quantifica-tion and propagation (Collopy et al. 2012; Lewis andMistree 1998; Simpson and Martins 2011; Tolson andSobieszczanski-Sobieski 1985), and topology optimiza-tion (Allison and Herber 2013; Simpson and Martins2011). We believe that DG has the capability to providea solid theoretical foundation for making these incor-porations. The relevant mathematical theory alreadyexists; it is simply a question of translating and apply-ing that theory. Such work would likely require more

DG than has been described here, but those additionswould build upon our work here, not replace it.

6 Present framework limitations

Having touched on some of our framework’s capabil-ities, we now consider its limitations under two maincategories: constraint management and the calculationand differentiation of the relevant quantities.

6.1 Constraints

Constraint-related concerns can, at this point, come un-der two headings: inequality constraints and additional(non-state) equality constraints.

Inequality constraints define the boundary ofMfeas,notMfeas itself, but they tend to play an important rolein constrained optimization because optimal solutionsoften lie on the boundary of the feasible set. We havenot yet explicitly taken them into account, however.One option for handling them would be to turn eachinequality into an equality through the use of a slackvariable and then consider the slack variables as artifi-cial state variables. This seems less than ideal, however,for two reasons. Firstly, it is not effective at handling ex-plicit bounds on variables: for example, if the slack vari-ables s were implemented as g (w,y)+ s = 0, all of theslack variables would still be required to satisfy s % 0,and in any case, explicit bounds on the existing stateand design variables would be left untouched. Thesevariables’ bounds define a manifold boundary just asmuch as the inequality constraints, albeit in a simplerway, so it seems that little is gained by this approach.

Secondly, although there may be relatively few stateequations in a given design problem, there are oftenmany more inequality constraints, so handling thoseconstraints in this way would turn a low-dimensionalfeasible design manifold into a very high-dimensionalmanifold composed mainly of slack variables. Increas-ing the dimension of the design space in this way iscounterproductive to the whole endeavor of seeking asimplifying analysis tool. Another option would be tojust consider the active set as equalities (thereby defin-ing a pertinent submanifold of "Mfeas). However, theactive set would be changing throughout the processof optimization, and changes in the active set wouldchange the nature and properties of the submanifoldof "Mfeas in question, thus changing the analysis toolslike gij , so this may not be helpful.

We have also not considered equality constraints be-yond the state equations, but it is possible to have addi-tional equality constraints which do not correspond to

10 Craig Bakker, Geo!rey T. Parks

a disciplinary output. The solutions to these additionalconstraints, assuming that the state equations were sat-isfied as well, would define a submanifold of the mani-fold defined by the state equations. It would be possibleto define a metric (with the requisite derivatives) forthis submanifold using implicit function and submani-fold theorems, but then which design variables shouldbe considered as dependent variables for the additionalequations? The state equations naturally have the statevariables as dependent variables, but now a decisionhas to be made. A designer could take a set of designvariables (presumably but not necessarily global designvariables) and redefine them to be dependent variables,but then those variables would effectively be state vari-ables, and the situation would just revert to the orig-inally defined design problem with the only equalityconstraints being state equations; this would be effec-tively the same as re-labeling some of the wi’s as yi’s.In this case, there is no loss in generality to simply as-sume that there are no equality constraints other thanthe state equations. It is difficult to see how else theequality constraints might be profitably dealt with.

The application of Value-Driven Design (VDD) toMDO may be worth considering here (Collopy et al.2012). VDD removes design emphasis from constraintsand places it on the objective function: “While it isrecognized that in some industries a complete elimi-nation of constraints is not possible, VDD’s goal is toeliminate as many as possible, incorporating those pref-erences communicated through the requirements intothe value function.” (Mesmer et al. 2013, p.10) If VDDis as valuable an addition to MDO as its proponentsclaim, and if VDD were to gain traction in MDO circles,then the additional constraints discussed above wouldbecome less important or even non-existent, and thusthe issues identified in this section would become cor-respondingly less important or non-existent.

There is a connection here to the optimization prob-lem’s Lagrangian as well. Given the reconceptualizationof the constrained optimization problem as an uncon-strained problem on a Riemannian manifold (see thebeginning of Section 3), the resulting Lagrangian (onthe Riemannian manifold) could have inequality con-straint terms, but it would not have equality terms cor-responding to the state equations because those con-straints would have already been “absorbed”, for lackof a better term, into the Riemannian manifold and itsstructure; the additional non-state equality constraintsmentioned earlier could show up in the Lagrangian,though. We reiterate, though, that part of the pointof this framework is to facilitate seeing MDO as uncon-strained optimization (or at least optimization without

equality constraints) on Riemannian manifolds ratherthan simply as constrained optimization.

6.2 Calculability and differentiability

Throughout this analysis, we have tacitly assumed thatfor a given w there is a unique solution y to h (w,y) =0. That is to say, there exists a point-to-point mapping

hy (w) = {y|h (w,y) = 0} (35)

This need not be the case, however, and it is an is-sue that has been addressed in the similar context ofBilevel Programming Problems (Bard 1991). If (35) isin fact a point-to-set mapping, it may have differentia-bility problems; if each y is locally unique, that shouldnot be a problem – all of the tensor quantities calcu-lated so far are strictly local – but if not, it is uncertainwhat would happen. It effectively hinges on "h

"y beinginvertible: we require it to be invertible in order to cal-culate the quantities described in Section 4.

The previous analysis has also assumed that themanifolds are sufficiently smooth to allow all of the op-erations that have been performed on them, and, more-over, that the Riemannian structure can be imposedon such a sufficiently smooth manifold. Every smoothmanifold admits a Riemannian metric (Boothby 1986),but there are real-world design problems that are notsmooth. The use of metamodelling in MDO becomespertinent at this point: the metamodels often used toapproximate those problems during optimization tendto smooth out the design space in the process of approx-imation (Sobieszczanski-Sobieski and Haftka 1997).

Furthermore, according to the Weierstrass theorem,any continuous function can be approximated to arbi-trary precision by a smooth function (Boothby 1986).For our purposes, this means that even though we areonly considering smooth design manifolds, those mani-folds can be considered as arbitrarily good approxima-tions to all continuous design manifolds. Thus the re-striction is less severe than might be otherwise thought.Since, as previously noted, the tensor quantities beingcalculated are all local, the smoothness required of anypoint on the manifold would only have to be local, notglobal, too. If there are discontinuities in the manifold,however, those discontinuities could cause problems inways similar to local non-uniqueness as discussed above.

7 Conclusions

The mathematics behind DG can initially appear some-what intimidating, but once given concrete form in their

Di!erential Geometry Tools for Multidisciplinary Design Optimization, Part I: Theory 11

application to MDO, they actually prove to be quitetractable. The index notation, in particular, simplifiesthe bookkeeping involved, and the resulting calcula-tions end up being less intimidating combinations ofcalculus and algebra. Visualizing high-dimensional man-ifolds is also notoriously difficult, but the mathematicshere make it possible to work without needing directgeometric intuition into a given problem. We have else-where already used some of the tools provided by ourframework; in Part II we will provide an example ofhow some of those tools can be used to investigate andimprove the performance of QSD.

Acknowledgments

This research is supported by the Natural Sciences andEngineering Research Council of Canada (NSERC) andthe Cambridge Commonwealth Trust and CambridgeOverseas Trust (CCT/COT).

References

Agte J, de Weck O, Sobieszczanski-Sobieski J, Arend-son P, Morris A, Spieck M (2010) MDO: Assessmentand direction for advancement - an opinion of one in-ternational group. Structural and MultidisciplinaryOptimization 40:17–33

Alexandrov NM, Lewis RM (2000) Algorithmic per-spectives on problem formulations in MDO. In:8th AIAA/NASA/ISSMO Symposium on Multidis-ciplinary Analysis, AIAA, Long Beach, California

Alexandrov NM, Lewis RM (2002) Analytical and com-putational aspects of collaborative optimization formultidisciplinary design. AIAA Journal 40:301–309

Alexandrov NM, Lewis RM (2003) Dynamically recon-figurable approach to multidisciplinary problems. In:16th AIAA Computational Fluid Dynamics Confer-ence, AIAA, Orlando, Florida

Allison JT, Herber DR (2013) Multidisciplinary de-sign optimization of dynamic engineering systems.In: 54th AIAA/ASME/ASCE/AHS/ASC Structures,Structural Dynamics, and Materials Conference,AIAA, Boston, Massachusetts

Bakker C, Parks GT (2014) Multi-objective optimiza-tion, MDO, and differential geometry controlledpareto front solution spacing. In: 4th InternationalConference on Engineering Optimization, ISSMO,Lisbon, Portugal

Bakker C, Parks GT, Jarrett JP (2012) Geometricperspectives on MDO and MDO architectures. In:12th Aviation Technology, Integration and Opera-tions (ATIO) Conference and 14th AIAA/ISSMO

Multidisciplinary Analysis and Optimization Confer-ence, AIAA, Indianapolis, Indiana

Bakker C, Parks GT, Jarrett JP (2013a) Differentialgeometry and design coupling in MDO. In: 54th

AIAA/ASME/ASCE/AHS/ASC Structures, Struc-tural Dynamics, and Materials Conference, AIAA,Boston, Massachusetts

Bakker C, Parks GT, Jarrett JP (2013b) Optimiza-tion algorithms andODE’s inMDO. In: ASME 2013Design Engineering Technical Conferences and Com-puters and Information in Engineering Conference,ASME, Portland, Oregon

Bard JF (1991) Some properties of the bilevel program-ming problem. Journal of Optimization Theory andApplications 68:371–378

Barthelemy JFM, Sobieszczanski-Sobieski J (1983a)Extrapolation on optimum design based on sensitiv-ity derivatives. AIAA Journal 21:797–799

Barthelemy JFM, Sobieszczanski-Sobieski J (1983b)Optimum sensitivity derivatives of objective func-tions in nonlinear programming. AIAA Journal21:913–915

Bloebaum CL, Hajela P, Sobieszczanski-Sobieski J(1992) Non-hierarchic system decomposition instructural optimization. Engineering Optimization19:171–186

Boothby WM (1986) An Introduction to Differen-tiable Manifolds and Riemannian Geometry. Aca-demic Press, Inc., Boston

Braun RD, Kroo IM (1997) Development and applica-tion of the collaborative optimization architecture ina multidisciplinary design environment. In: Alexan-drov N, Hussaini M (eds) Multidisciplinary DesignOptimization: State of the Art, SIAM, Philadelphia,pp 98–116

Collopy PD, Bloebaum CL, Mesmer BL (2012) Thedistinct and interrelated roles of value-driven de-sign, multidisciplinary design optimization, and de-cision analysis. In: 12th Aviation Technology, Inte-gration and Operations (ATIO) Conference and 14th

AIAA/ISSMO Multidisciplinary Analysis and Opti-mization Conference, AIAA, Indianapolis, Indiana

Cramer E, Dennis JE Jr, Frank PD, Lewis RM, ShubinGR (1994) Problem formulation for multidisciplinaryoptimization problems. SIAM Journal on Optimiza-tion 4:754–776

DeMiguel AV, Murray W (2000) An analysisof collaborative optimization methods. In: 8th

AIAA/NASA/ISSMO Symposium on Multidisci-plinary Analysis, AIAA, Long Beach, California

DeMiguel V, Murray W (2006) A local convergenceanalysis of bilevel decomposition algorithms. Opti-mization and Engineering 7:99–133

12 Craig Bakker, Geo!rey T. Parks

Giesing JP, Barthelemy JFM (1998) A summary ofindustry MDO applications and needs. In: 7th

AIAA/NASA/ISSMO Symposium on Multidisci-plinary Analysis and Optimization, AIAA, St. Louis,Missouri

Haftka RT (1985) Simultaneous analysis and design.AIAA Journal 23:1099–1103

Haftka RT, Watson LT (2005) Multidisciplinary designoptimization with quasiseparable subsystems. Opti-mization and Engineering 6:9–20

Haftka RT, Watson LT (2006) Decomposition theoryfor multidisciplinary design optimization problemswith mixed integer quasiseparable subsystems. Op-timization and Engineering 7:135–149

Han J, Papalambros PY (2010) A note on the con-vergence of analytical target cascading with infinitenorms. Journal of Mechanical Design – Transactionsof the ASME 132:034,502

Ivancevic VG, Ivancevic TT (2007) Applied DifferentialGeometry: A Modern Introduction. World ScientificPublishing Co. Pte. Ltd., Singapore

van Keulen F, Haftka RT, Kim NH (2005) Review of op-tions for structural design sensitivity analysis. part 1:Linear systems. Computer Methods in Applied Me-chanics and Engineering 194:3213–3243

Kim HM (2001) Target cascading in optimal system de-sign. PhD thesis, University of Michigan, Ann Arbor,Michigan

Kim HM, Chen W, Wiecek MM (2006) Lagrangian co-ordination for enhancing the convergence of analyti-cal target cascading. AIAA Journal 44:2197–2207

Lewis K, Mistree F (1998) The other side of multidis-ciplinary design optimization: Accomodating a mul-tiobjective, uncertain and non-deterministic world.Engineering Optimization 31:161–189

Martins JRRA, Hwang JT (2012) Review andunification of methods for computing deriva-tives of multidisciplinary systems. In: 53rd

AIAA/ASME/ASCE/ASC Structures, Struc-tural Dynamics, and Materials Conference, AIAA,Honolulu, Hawaii

Martins JRRA, Lambe AB (2013) Multidisciplinarydesign optimization: Survey of architectures. AIAAJournal 51:2049–2075

Mattson CA, Mullur AA, Messac A (2004) SmartPareto filter: Obtaining a minimal representation ofmultiobjective design space. Engineering Optimiza-tion 36:721–740

Mesmer BL, Bloebaum CL, Kannan H (2013) Incorpo-ration of value-driven design in multidisciplinary de-sign optimization. In: 10th World Congress on Struc-tural and Multidisciplinary Optimization, ISSMO,Orlando, Florida

Michelena N, Park H, Papalambros PY (2003) Conver-gence properties of analytical target cascading. AIAAJournal 41:897–905

de Miguel AV (2001) Two decomposition algorithms fornonconvex optimization problems with global vari-ables. PhD thesis, Stanford University, Stanford, Cal-ifornia

Penrose R (1989) The Emperor’s New Mind. OxfordUniversity Press, Oxford, England

Roth BD, Kroo IM (2008) Enhanced collaborative opti-mization: A decomposition-based method for multi-disciplinary design. In: ASME 2008 Design Engineer-ing Technical Conferences and Computers and Infor-mation in Engineering Conference, ASME, Brooklyn,New York

Simpson TW, Martins JRAA (2011) Multidisciplinarydesign optimization for complex engineered systems:Report from a National Science Foundation work-shop. Journal of Mechanical Design – Transactionsof the ASME 133:101,002

Sobieszczanski-Sobieski J (1990a) Sensitivity of com-plex, internally coupled systems. AIAA Journal28:153–160

Sobieszczanski-Sobieski J (1990b) Higher order sensi-tivity analysis of complex coupled systems. AIAAJournal 28:756–758

Sobieszczanski-Sobieski J, Haftka RT (1997) Multidis-ciplinary aerospace design optimization: Survey of re-cent developments. Structural Optimization 14:1–23

Sobieszczanski-Sobieski J, Agte JS, Sandusky RR Jr(1998) Bi-level integrated system synthesis (BLISS).Tech. Rep. NASA/TM-1998-208715, NASA LangleyResearch Center, Hampton, Virginia

Szekeres P (2004) A Course in Modern Mathemati-cal Physics. Cambridge University Press, Cambridge,England

Tolson RH, Sobieszczanski-Sobieski J (1985) Multidis-ciplinary analysis and synthesis: Needs and oppor-tunities. In: 26th AIAA/ASME/ASCE/AHS Struc-tures, Structural Dynamics, and Materials Confer-ence, AIAA, Orlando, Florida

Tosserams S, Etman LFP, Papalambros PY, Rooda JE(2006) An augmented Lagrangian relaxation for an-alytical target cascading. Structural and Multidisci-plinary Optimization 31:176–189

Tosserams S, Etman LFP, Rooda JE (2009) Block-separable linking constraints in augmented La-grangian coordination. Structural and Multidisci-plinary Optimization 37:521–527

Yano K, Kon M (1984) Structures on Manifolds. WorldScientific Publishing Co. Pte. Ltd., Singapore