COLLECTIVES AND COMPLEX SYSTEM DESIGNaero.stanford.edu/Reports/VKI_Collectives_Kroo_A.pdf · COLLECTIVES AND COMPLEX SYSTEM DESIGN ... the application of optimization in the design

VKI lecture series on Optimization Methods & Tools for Multicriteria/Multidisciplinary Design November 15-19, 2004

1

COLLECTIVES AND COMPLEX SYSTEM DESIGN

I. Kroo

Stanford University, U.S.A.

1. Summary A collective is a group of self-motivated agents that maximize system performance through the pursuit of local objectives. This paper deals with two aspects of collective design: the application of optimization in the design of collectives, and the use of collectives for engineering design itself, in which multiple individuals or teams design parts of a large-scale system. Examples are drawn from aeronautical engineering, where well-developed models in the many relevant disciplines may be used to analyze and optimize the complete system. The design of aircraft, involving many individuals or organizations, and the formation flight of geese, for example, share many similar features. In each case, individuals must decide on a course of action that must benefit the system as a whole, despite the requirement that they act locally and cannot immediately ascertain the effect of their actions on the entire system. Examples demonstrate how multi-level distributed optimization can be used to achieve optimal system performance while focusing on local degrees of freedom and how a similar approach leads to the optimal V-shaped flock of geese, even when individual birds seek only to maximize their own local goals. Additional applications of these ideas show how collectives may provide new engineering solutions to problems in aerospace design. 2. Introduction A variety of interesting problems in aeronautics involve the interaction among multiple agents that must act as a group to accomplish some task. These agents may be unmanned air vehicles searching cooperatively, a group of control actuators that must act in concert to achieve a desired response, or a team of design experts working on portions of an overall system design problem. Indeed this type of problem is common throughout nature and society, from insect colonies and distributed corporate organizations to national economies. And although design techniques for individual vehicles or controllers are well-developed, design strategies and tools for the optimization of multi-agent systems are often primitive and heuristic, despite the importance of these systems in engineering and society. In this paper we consider a special type of multi-agent system, a “collective” in which a group of self-motivated individuals seek to maximize the performance of the system as a whole. This paper describes the emerging theory of collectives, approaches to their design, and their application in aeronautical engineering. The development of aeronautical systems has focused on the design of individual vehicles, which are then sometimes assembled into a fleet, to become the air transportation system, for example. As the complexity of these systems grows, however, this approach to system design becomes more difficult and less optimal. New theories of collective behavior, an improved understanding of emergent system properties, and new


2

approaches to the design of multi-agent systems promise to significantly change the way that aeronautical systems are developed. Whether the system of interest involves network routing (of aircraft or data packets), the distributed design of a complex system by multiple disciplinary design teams, or the coordination of multiple air vehicles for performance enhancement or air traffic management, the science of multi-agent systems can be applied to create a system of systems whose performance may greatly exceed that of an ad-hoc aggregated system. The design or control of a complex, multi-agent system is difficult because important aspects of the system’s performance may be emergent properties of the system. Although much work has been done in the field of complex systems theory and many examples of system-level emergent behavior arising from simple local rules are well-known [1,2], we are interested not in interesting system dynamics, but in optimal emergent behavior. And the two conventional approaches to system design are problematic in these cases, since decomposition of the problem into more tractable subsystem design problems misses the emergent system properties, while centralized optimization is infeasible because of the complexity of the problem. In contrast to a centralized optimization approach or a hierarchical problem decomposition, a collective is a kind of distributed optimization system in which individual agents seek to maximize a local utility, while interacting with other agents who are simultaneously choosing actions that increase their own local utility. Formulating the problem as a distributed optimization allows the use of techniques from machine learning, statistics, multi-agent systems, and game theory. The emerging field of “collectives” or “collective intelligence” leverages results in these related fields, supplies a framework for designing a collective, and has been applied to a variety of distributed optimization problems including network routing, computing resource allocation, and data collection by autonomous rovers. [3-5]. Two basic problems exist in the field of collectives. Predicting the resulting dynamics of the system, its equilibrium points, and the system performance (global utility) constitutes the forward problem. This may be accomplished using simulation (fictitious play), experiment, or, in some cases, probabilistic or aggregation theories. The inverse problem of collectives is usually the more difficult problem and the focus of the present discussion. The problem is to determine the individual agents’ local utilities and the strategies that they use to select actions that maximize their utilities in such a way that the overall system utility is maximized. 3. Fundamentals of Collectives In a collective design process, agents select actions (a value from the variable space) and receive rewards that must be based in some way based on the system objective. These rewards are then used by the agents to determine their next choice of action. The process


3

reaches equilibrium when the agents can no longer improve their rewards by changing actions. The inverse problem of collectives is a kind of meta-design problem and centers around two fundamental questions:

1. How does one select local utilities for individual agents that, when pursued in a distributed system, leads to the desired system performance?

2. What strategies should be followed by the individual agents to efficiently lead to increasing their local utilities?

3.1. Local or Private Utilities In some systems the choice of local utility functions is specified by the character of the system itself. Thus in economic systems, agents are self interested with their own, sometimes unknown goals. These may be influenced by incentives and the field of mechanism design is closely related to the problem of utility selection in collectives, but is more restrictive. In the case of collective design, we are free to choose the utility functions that agents seek to maximize. This choice is not obvious, however, and poor choices can lead to very inefficient or poorly performing collectives. The theory of collectives described by Wolpert and Tumer [6] and the more recent probability collectives theory [7] provide some formal considerations that assist in this choice. This approach is outlined in the following sections. In all of the cases of interest here, there exists a global evaluation function or system utility, G(z), which is a function of all of the environmental variables and the actions, z, of all the agents. The goal of the collective is to maximize G(z); however, the agents do not maximize G(z) directly. Instead each agent, i, works to maximize its private evaluation function gi(z). The problem for a designer of such a collective is to select the g(z)’s so that maximizing all of the g(z)’s also leads to maximizing G(z). 3.1.1. Factoredness and Learnability Two properties of the local utilities are necessary for a collective to achieve good values of the global utility, G. The first property, well-known in game theory, is that the local utilities must be aligned with, or factored with respect to, the global utility. That is, an action taken by an agent that improves its utility should also improve the system utility. Formally g is factored when: gi (z) ≥ gi (z’) ⇔ G(z) ≥ G(z’) ∀z, z’ s.t. z-i = z’-i . where z-i and z’-i contain the components of the sets of actions, z and z’ respectively, that are not influenced by agent i. If agents pursue selfish goals that are not factored with respect to the global utility, classic difficulties, such as the tragedy of the commons or Braess’ paradox [4] are common. Yet, it is easy to create local utilities that are factored. The most obvious is that associated with a “team game”, in which all agents use the same local utility function, namely the global utility function: gi = G.


4

A common difficulty with the use of team games in large scale collectives is that individual agents may have difficulty determining the influence of their action on the global utility. In a large corporation, for example, tying employee compensation to market valuation of the company leads to a factored system, but an individual employee usually cannot tell what effect his or her action might have on stock prices. This sensitivity to individual agent actions and insensitivity to the actions of others is a kind of signal to noise ratio, termed “learnability” in the collective intelligence literature. It is defined quantitatively as [6]: λi,gi (ζ) ≡ || ∇ζi gi (ζ)|| / ||∇ζ−i gi(ζ)|| and is meant to measure the sensitivity of gi (z) to changes to i’s actions, as opposed to changes to other agent’s actions. 3.1.2. Difference Utilities One approach to creating factored, learnable local utility functions is to create difference utility function that are related to the global utility function by: gi ≡ G(z) − G(z-i + ci) where z-i contains all the variables not affected by agent i. Difference utilities of this form are factored for any choice of the constant ci, because the second term does not depend on i’s actions [6]. The learnability is also enhanced with respect to the team game utility, since the differencing removes much of the effect of other agents (noise) on each agent’s utility function. As noted in [8]: “In many situations it is possible to use a ci that is equivalent to taking agent i out of the system. Intuitively this causes the second term of the difference evaluation function to evaluate the fitness of the system without i and therefore g evaluates the agent’s contribution to the global evaluation.” Several types of difference utilities have been investigated [5]. These include the “Wonderful Life Utility” (WLU) in which the value of the subtracted term in gi is computed by ignoring the effect of the ith agent on the system, and several other utilities created by fixing or “clamping” the ith agent’s action to a prescribed value. The actual value used for the clamping operation does not affect the factoredness of the system but does have an effect on the convergence rate of typical test problems[5]. 3.2. Implementation 3.2.1. Direct optimization Once the definition of local utilities has been determined, individual agents work to maximize their local utilities. This has been accomplished in a number of ways including gradient-based optimization, evolutionary methods [8], and reinforcement learning [5]. One of the issues in efficient implementation of a collective involves how individual agents compute their local utilities. In some cases, it is difficult to compute the global utility, G. Depending on the function, it may also be difficult or impossible to compute the “counter-factual” part of the difference utility: what would the world have been like without me (in the “wonderful life” utility, for example). In these cases it is often possible to estimate the value of G or g based on the information that is available [9]. A second item of interest involves convergence of the collective optimization process. Although this is difficult to analyze in general because of the range of different


5

optimization techniques that may be used and the multiple-level process involved (selection, or rewards, or other data-driven update for each agent as the system evolves), some general results may be applicable [10] and for certain types of update methods (e.g. probability collectives) additional results from game theory may be applied. 3.2.2. Probability Collectives Recently, a variation on the collective intelligence concept has been proposed [7] that replaces the optimization across possible actions with one across the probability space of possible actions. This “probability collectives” theory is based on concepts from bounded rationality game theory and information theory. Wolpert et al. [11] show how when the probability of an agent’s action is regarded as the unknown, rather than the action itself, the system objective function may be replaced with a function of the probabilities (the Lagrangian), and the system problem is to minimize the following function with respect to the unknown probability function p(z): L(p) = Σ p(z) E [G(z)] + T Σ p(z) ln p(z) Where E[G] is the expectation value of the global objective, G, and T is a temperature. The sum is taken over all of the possible joint actions, z. The Lagrangian is composed of a term that reflects the expected reward across the actions and the entropy associated with the probabilities of those actions. As the Lagrangian is minimized, temperature, T, is used to trade off the exploitation of good actions (favored by lower temperatures) with exploration of other possible actions (encouraged by higher temperatures). Replacing actions or design variable values with the probability of an agent selecting those values seems an unnecessary complication to an already difficult problem, However, there are several reasons that this might make the problem more tractable. The formulation has connections to other fields such as game theory, statistical mechanics, and gradient-based optimization, and results from these fields may be used to suggest solution strategies. For certain types of global objective functions (e.g. G = Σ gi ) simple solutions exist for the individual agent updates. In addition the formulation may be used directly for functions that are inherently probabilistic, or discontinuous, or in cases for which the design variables must be discretized (since the probability functions remain continuous). Known or easily computed problem structures, such as variable dependencies or hierarchies, can be exploited in this formulation as well. The problem is simplified by approximating the joint probability distribution, p(z) as a product of probability distributions, qi(zi) -- one for each agent. This allows agents to select actions independently, but distributions are still coupled in the evaluation of the expectation value of G. Introducing a local objective, gi, that may consist of the global utility, or a related difference utility as described previously, the ith agent seeks to find the probability distribution, qi, that minimizes: Li (qi) = Σ qi(zi) E [gi (zi, z-i) | zi ] + T Σ qi(zi) ln qi(zi)


6

This local Lagrangian has many nice features (convex, single minimum in the interior) and has an analytical gradient (and Hessian). The gradient and Hessian are used to obtain an update rule which has a simple form and is easy to implement [11]. The expectations can either be estimated using Monte-Carlo sampling or computed analytically. The estimation introduces error which may be addressed by the choice of private utility. In the parlance of probability collectives, the variance is related (inversely) to learnability or signal-to-noise ratio, while bias provides a quantitative estimate of factoredness. A summary of this process is shown in figure 1. Each agent conducts sampling to estimate the expectation value of its utility function. This function is fit using a regression model and this model is used to update the agent’s probability distribution. The process is repeated, generally with decreasing temperature until the probability distributions are sufficiently “peaky” that the system objective no longer changes.

Figure 1. Probability collectives algorithm for collective design (from [11]).


7

4. Collective Design: Aeronautical Applications Aeronautical systems provide interesting examples of collective behavior, from the design process itself, in which many individuals or teams design a part of a large scale system whose parts must work in harmony, to the operation and control of these complex, multi-component systems. Interactions among individual components, teams, or vehicles in aeronautics, is often strong and based on complex physics that is nonetheless amenable to computational modeling. Although 100 years ago, simple aircraft could be designed by one or two people, current designs require teams of experts and no one understands every aspect of the system. Such examples have therefore proven quite useful in many fields including computation-based design, multidisciplinary optimization, and multi-agent control. In these notes, recent results from these fields are applied to example systems that illustrate some of the fundamental issues in collective system design. The goal is to identify scalable, non-heuristic approaches to distributed system design, focusing on high-level control and planning rather than the details of the dynamic response. Although the theory of collectives and strategies for the design of complex systems are still in their infancy, several striking examples of the potential for this approach have been described recently [12] in other fields and further progress is likely to spur innovation in aeronautical systems for decades to come. The following examples are intended to provide only a suggestion for how this science could change future aeronautical design concepts. 4.1. Collectives by Design Simple difference utilities or probability-based Lagrangians are reasonable choices for local objectives in a collective, but through multiple simulations it is possible to address the meta-design problem of collectives directly. The following recent aeronautical examples illustrate the use of off-line optimization to determine the goals that individuals in a collective should pursue so as to produce the desired group behavior. These examples include the collective control of a flexible wing with distributed micro-flaps, and decentralized optimization for efficient formation flight of a number of air vehicles (or birds).


8

4.1.1. Control Using Distributed Micro-flaps An unconventional application of the idea of collective systems to vehicle flight control is illustrated by the distributed control system shown in figure 2. The MiTE concept involves replacing or augmenting conventional control surfaces with a large number of simple and small trailing edge devices as shown below.

Figure 2. Distributed control using small trailing edge effectors.

The small surfaces (1% to 4% of the chord) are deflected in one of three states: neutral, up, or down, eliminating the need for servo feedback to accurately position the surfaces. Because of their small size, MiTEs provide very high bandwidth control with low power requirements. One difficulty with these devices is that they exhibit very nonlinear behavior due to the manner in which they are deflected, causing vortex formation near the trailing edge as shown in the time sequence of figure 3. This represents a difficulty when synthesizing control laws for a group of perhaps hundreds of the devices.

Figure 3. Flow near trailing edge after deflection of MiTE. Sequence from Navier

Stokes simulation showing development of separation and shedding of vortex. The difficulty in synthesizing control laws for this system and the requirement for redundancy, which motivated the distributed nature of the system initially, suggests that


9

each flap might include a simple local sensor and processor to reduce communication, and be treated as a collective of individual agents that work together to control the wing, much as the cells of a sponge or slime mold work collectively. The control law design problem is then to create local utilities that are functions of the available sensed quantities and which if minimized by each agent (flap), leads to the desired behavior of the system.

Global

Sensors

Agents

Actuator

Local Sensors

Agent Controller

Agents

Single Agent

Figure 4. Conceptual view of blended-wing-body control using collectives.

An aeroelastic model of the blended-wing-body concept (figure 4) was created using linear aerodynamics and a finite element structural representation [13]. Numerical optimization was then used to find parameters in the agent’s local objective functions that led to the desired collective behavior. Results from this simulation are shown in figure 5. The lightly damped dynamic response of the open loop system is successfully damped when the distributed control is activated. The simulation excluded some of the time-dependent aerodynamics of the MiTE actuators and leads to a somewhat optimistic view of their effectiveness for aeroelastic control. Subsequent work included improved and more realistic models of actuators and sensors, based on wind tunnel experiments [14].

Figure 5. Simplified Aeroelastic Simulation Results: Wing Tip Deflection Time History


10

The design approach described in [15] uses reinforcement learning and the theory of collectives [6] to maximize the performance of a flexible wing with MiTE actuator system, while individual flaps make local decisions based on local information. The idea was demonstrated with a flexible-model wind tunnel test, in which MiTEs were fabricated and a distributed control law was created to maximize the flutter speed of an elastically tailored wing. This approach successfully suppressed flutter, increasing the allowable dynamic pressure by almost 50% [15].

Figure 6. Time-dependent lift and flow pattern for a rapidly-deflecting MiTE from time

accurate Navier Stokes simulation [16].

Figure 7. Time history of wing pitch rate and one of the actuator positions using

collective actuator flutter suppression. System is turned on at about t=77.5 seconds.


11

4.1.2. Formation Flight Perhaps a more obvious example of the potential advantages of group behavior is that of formation flight. As illustrated in the flocking behavior of many migratory birds and, as is well-known to aerodynamicists and pilots, substantial reductions in vortex drag may be achieved by exploiting favorable interference between two wings. Figure 8 (computed based on simple linear theory and subject to roll trim constraints) shows that when the two wing tips are separated laterally by a small distance, a vortex drag savings of about 40% may be achieved. Since the longitudinal spacing does not affect the total vortex drag, this close spacing does not require a dangerously close proximity between the tips. A similar savings is produced by a wing flying in formation with its own image (ground effect), when the distance above the symmetry plane is about 20% of the wing span.

Figure 8. Potential reductions in induced drag due to favorable interference

between two wings. When more than two aircraft fly in formation, the potential savings is much larger, with a very simple analysis suggesting that the average lift-to-drag ratio scales as the square root of the number of aircraft in the flock. Thus, the potential savings associated with even three or four aircraft in formation may far exceed that of wing configuration features such as winglets, which, although capable of reducing vortex drag, involve a structural weight penalty that sometimes offsets much of the advantage [17]. One of the outstanding problems with formation flight is maintaining the correct relative position of the aircraft in the formation. The success of good pilots in achieving precision formation flight shows that this is not a completely intractable problem, however, and recent work at NASA has investigated some of the critical aspects of autonomous formation flight [18]. This fundamentally different approach to aircraft


12

operation could be enabled by recent advances in precision navigation and increasingly capable and reliable automatic flight control. A second problem involves, not the details of precision station-keeping, but the design of the higher level strategy for efficient formation flight. With a large number of interacting air vehicles (or birds), simple rules can be created that lead, not just to the interesting looking emergent behavior described by Reynolds [19], but to a desired optimal system behavior. Viewed as a collective design problem the goal here is not to maintain a formation specified a priori but to optimize the performance of the group in a dynamic way that includes the important aerodynamic interactions among the birds. The specific problem is to identify local utilities and coordination strategies that lead to a robust solution. Each bird leaves a wake that influences the others in the flock. The drag of each bird consists of viscous drag, self-induced vortex drag, and interference effects that are modeled using linear aerodynamic theory. The simulation involves computing the effect of 20 discrete vortices in each bird’s wake on 20 points over the wing of each of the other birds as shown in figure 9.

Figure 9. Discrete vortex models are used to compute induced drag and interference

effects among all birds in the flock. The global objective in this example problem is to is maximize the range of a group of identical birds by minimizing the drag of the bird with the greatest drag -- a kind of “no bird left behind” concept. Each bird controls its individual speed, but knows nothing about aerodynamics and only measures the power required for it to fly at its selected speed. The problem might be solved directly by using centralized nonlinear optimization to find the best speed and position for each bird. This works well in the steady case for a limited size flock, and good initial distributions, but fails completely in other cases. It scales poorly, but provides a reference solution as shown in figure 10.


13

Figure 10. Distribution of birds that leads to maximum total range of the complete flock,

computed using gradient optimization. Viewed as a collective, the flock will maximize its performance when each individual seeks its own “best” solution. If each bird flies at the speed that minimizes its own drag (selfish solution), the resulting dynamics are unstable, as birds that trail others find that their optimum speed is slower than the leaders. Using an evolutionary algorithm and many simulations of the flock behavior, local utilities can be defined so that the collective does find an optimal system. The algorithm is as follows: Each bird, i, computes the speed that would minimize its own drag, Vi* at each step in the simulation, based on local observation. Each bird then selects its speed based on these local results and those of others: Vi = Σ Kij Vj* + Kii Vi*. With a simple symmetric coordination law: Vi = K1 Σ Vj* + K2 Vi* . K1 and K2 are determined using an evolutionary algorithm and multiple simulations. The result is not initially obvious, but the optimization leads to a negative value of K2 and, in fact, a value less than -K1, meaning that each bird’s speed should be more closely related to the speed that would minimize the drag of the others in the flock than its own. The solution is robust, leading to the correct system optimum for any initial condition, requiring limited communication among the individuals, and scaling easily to 100+ birds. The same form of local utility works for heterogeneous flocks and non-uniform wind conditions and may be of use in UAV cooperative control and future cargo aircraft formation flight concepts.


14

Figure 11. Evolution of optimal flock using collective control strategy (clockwise

starting from upper left). Figure 11 shows a top view of the time-based evolution of a group of 25 birds that start out with a random longitudinal distribution and by following a single simple rule and local measurements are able to efficiently and robustly find the desired solution.


15

Figure 12. “Artist’s” concept of formation flight for low cost cargo transportation. Use of configurations especially tailored for formation flight can amplify potential savings.

4.2. Design by Collectives In a sense, aerospace design is an ideal example of a successful collective. Although aircraft design as a single high-dimensionality nonlinear optimization problem is intractable, successful designs are achieved with a collective of self-motivated disciplinary design experts. However, the process is poorly understood. The objective of the present work is to identify robust strategies that are scalable and applicable to a wide range of problems. Centralized design using nonlinear programming is useful for many problems, but for large-scale distributed systems, efficiency and communication issues limit the scalability of this approach. One means by which conventional centralized optimization may be extended to larger scale problems is through the use of reduced basis modeling. This has been used successfully in aerodynamic and structural design, and current work in automatic identification of reduced basis models using the principle orthogonal directions ideas appears promising [20]. The alternative is a decentralized approach to the design of distributed systems. This idea has proven effective for some systems even when the design strategy has been completely heuristic or very domain-specific. Some of the more intriguing examples of this idea are found in self-organizing systems that follow simple rules yet achieve complex system-level behaviors. The present approach to the optimal design of distributed systems differs from some previous examples of self-organizing systems [19] in that the desired response must be more than just interesting with unexpected emergent properties. Rather the goal is to produce self-optimizing systems with desired emergent behavior. The approach described


16

here parallels that described by Wolpert et al. [11] as collective intelligence. In each case, local objectives and constraints (local utilities in COIN) are posited for individual agents, each of whom must discern an action based on this local measure of performance. One of the keys to the success of such methods is the selection of local objectives and the management of communication between agents so that while pursuing local goals, the interacting system is improved. In the examples here, individuals determine local actions based on nonlinear programming and the goal is to achieve not just improved system behavior, but rather an optimal system. The basic concept is to decompose the system design problem into many smaller design problems that are undertaken in parallel, while taking account of interactions among the constituents and constraints on communication. Some systems may be decomposed easily using a hierarchical scheme and weakly interacting systems may be solved in parallel with few problems. The objective, here, however is to identify a strategy that is effective for strongly interacting nonhierarchical systems as are common in aerospace design. A natural decomposition of the aircraft design problem, for example, is one that is used routinely in the aerospace industry, namely, decomposition along disciplinary boundaries. In this approach structural analysis and structural design is done by a structures team, while aerodynamicists design airfoil and wing geometries, and control system designers synthesize control laws. The difficulty with this approach is that responsibility for certain design decisions and system-level objectives and constraints must be shared in some way between the groups, since aerodynamic design decisions affect the structure. In fact, it is often said of these systems that, “Everything affects everything else.” When these interactions are not properly taken into account, poor decisions about the design are made as suggested in the (somewhat exaggerated) views of the design problem in figure 13.

Figure 13. Aircraft design as viewed by various disciplinary groups (from [21]).


17

Approaches to this problem have included a variety of multi-level optimization approaches described in [22-27] which have proven complicated, yet effective in certain applications. In these notes we consider the possible application of the theory of collectives to the large-scale distributed design problem. The goal is to create a decentralized, distributed optimization process that treats the system design team as a collective with discipline-based designers as the individual agents. As in multi-level decomposition, it is important that system performance is insensitive to the level of subspace convergence and that the process accommodates disciplinary constraint satisfaction and heterogeneous optimization methods to be used within the disciplines. The approach explored here uses the theory of probability collectives to manage interactions among agents, but retains the discipline-centric constrained optimization methods for the agents themselves. The overall architecture is shown in figure 14 and resembles that of collaborative optimization [28] but with the central, system-level optimizer replaced by implicit coordination using collectives (PD theory).

Figure 14. Basic structure of collective-based distributed design system.

Since this strategy for design decomposition has not been investigated previously, initial studies of its feasibility were based on a simple, analytic problem that has been studied previously in connection with multi-level design [29]. The simple two-agent test problem is shown in figure 15 and consists of a quadratic global objective that depends on the actions of each agent. The problem is coupled with a linear constraint that must be satisfied by each agent, but that depends (in an adjustable way) on the other agent.


18

Figure 15. Constrained quadratic test problem with adjustable coupling (Shankar) used

to evaluate distributed optimization strategies. Although this problem is best solved with a single, conventional optimization algorithm, it serves as a useful test of the collective coordination idea when decoupled as shown in figure 16.

Figure 16. Architecture for collective solution of the decomposed Shankar problem.

Figure 17 shows the iteration history of a probability collective used to solve this problem. Following the approach outlined in section 3.2.2, and described in more detail in [11] the probability distribution of the values of z1 and z2 evolve from broad, uncertain values to sharply peaked distributions centered around z1 = 0.8, z2 = 1.6, the correct solution to the problem with β = 0.5. Note that because of the generic sampling method chosen, each iteration involves far more function evaluations (optimization problems in the subspaces) than required by alternative methods such as collaborative optimization, but the system does successfully identify the interactions between the agents, even with the high degree of coupling used to compute the results of figure 17. The challenge for future applications of this idea is to incorporate additional knowledge about the problem structure to improve the efficiency of this decentralized approach to design.


19

Figure 17. Evolution of probability distributions for each agent

over 30 iterations of the collective.


20

5. Conclusions and Future Work Many useful applications of collectives are apparent in aerospace design problems. The aeroelastic control of wings with distributed micro-flaps and the emergent optimal flocking behavior of birds seeking to minimize individual objectives illustrate how collectives may be designed using numerical optimization. Similarly, the collective concept may be applied to large-scale multidisciplinary design problems in which teams of designers work to improve the system by maximizing a local measure of performance that leads to optimal behavior of the complete system. The theory of probability collectives shows promise for these problems, but will require additional development to achieve the efficiency of other distributed design strategies. Future work may include continued studies of systematic approaches to generation of local utilities and the application of collectives to other problems including coordination of autonomous aircraft, air traffic control, design with uncertainty, and system-of-systems design problems. 6. Acknowledgments Much of the work summarized here has been accomplished by doctoral students in the Aircraft Aerodynamics and Design Group at Stanford (Stefan Bieniawski, Hak-Tae Lee, and Mike Holden). Interested readers should consult their papers and theses listed in the references section for further details. Most of the fundamental ideas underlying collectives and collective intelligence have been developed by David Wolpert, Kagan Tumer, and Bill Macready at NASA’s Ames Research Center. The author would also like to gratefully acknowledge support from the Boeing Company and NASA’s Ames and Langley Research Centers and the always stimulating interactions with colleagues there and at Stanford. 7. References [1] Bar-Yam,Yaneer, Dynamics of Complex Systems, Addison-Wesley, 1997. [2] Wolfram, S., A New Kind of Science, Wolfram Media, Champaign, IL, 2002. [3] Tumer, K., Agogino A., Wolpert, D. H., “Learning sequences of actions in collectives of autonomous agents,” In Proceedings of the First International Joint Conference on Autonomous and Multi-Agent Systems, Bologna, Italy, July 2002. [4] Wolpert, D. H., Tumer, K., “Collective Intelligence, Data Routing, and Braess' Paradox,” Journal of Artificial Intelligence Research, 2002. [5] Wolpert, D.H., Wheeler, K., Tumer, K., “Collective Intelligence for Control of Distributed Dynamical Systems,” Europhysics Letters, vol. 49 issue 6, 708-714, 2000.


21

[6] Wolpert, D. H., Tumer, K.. “An Introduction to Collective Intelligence.” Technical Report NASA-ARC-IC-99-63, NASA Ames Research Center, 1999. URL:http://ic.arc.nasa.gov/ic/projects/coin pubs.html. To appear in Handbook of Agent Technology, Ed. J. M. Bradshaw, AAAI/MIT Press. [7] Wolpert, D.H., “Information theory - the bridge connecting bounded rational game theory and statistical physics”, in Complex Engineering Systems, D. Braha, Ali Minai, and Y. Bar-Yam (Editors), Perseus books, in press. [8] Agogino, A., Tumer, K., “Efficient Evaluation Functions for Multi-Rover Systems,” Genetic and Evolutionary Computation Conference, Seattle, WA, June 26-30, 2004. [9] Lee, C.F., Wolpert, D.H., “Product distribution theory for control of multi-agent systems”, In Proceedings of the Third International Joint Conference on Autonomous Agents and Multi-Agent Systems, New York, NY, July 19-23, 2004. [10] Shamma, J.S., Arslan, G., "Unified convergence proofs of continuous-time fictitious play", IEEE Transactions on Automatic Control, July 2004, pp. 1137--1142. [11] Bieniawski, S., Wolpert, D.H., Kroo, I., “Discrete, Continuous, and Constrained Optimization Using Collectives,” AIAA Paper 2004-4580, Presented at the 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Albany, NY, August 30-September 1, 2004. [12] Tumer, K., “Applications of Collectives,” Tutorial Session, in Collectives and the Design of Complex Systems, CDOCS03, in press. [13] Holden, M., E., “Optimization of Dynamic Systems Using Collocation Methods,” Ph.D. Dissertation, Stanford University, 1999. [14] Solovitz, S., “Experimental Aerodynamics of Mesoscale Trailing-Edge Actuators, “ Ph.D. Thesis, Stanford University, 2002. [15] Bieniawski, S., Kroo, I., “Flutter Suppression Using Micro-Trailing Edge Effectors,” AIAA-2003-1941, Presented at 44th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Norfolk, Virginia, Apr. 7-10, 2003. [16] Lee, H., Kroo, I., “Computational Investigation of Airfoils with Miniature Trailing Edge Control Surfaces,” AIAA-2004-1051, Presented at 42nd AIAA Aerospace Sciences Meeting and Exhibit, Reno, Nevada, Jan. 5-8, 2004. [17] Jones,R.,Lasinski,T.,"Minimum Induced Drag of Wings with Winglets", NASA TM-81230, Sept. 1980.


22

[18] Wagner, G., et al. "Flight Test Results of Close Formation Flight for Fuel Savings." AIAA paper 2002-4490 , Monterey CA, Aug 2002. [19] Reynolds, C. W., “Flocks, Herds, and Schools: A Distributed Behavioral Model,” in Computer Graphics, 21(4) (SIGGRAPH '87 Conference Proceedings) 1987, pages 25-34. http://www.cs.toronto.edu/~dt/siggraph97-course/cwr87/ [20] LeGresley, P., Alonso, J., “Improving the Performance of Design Decomposition Methods with POD,” AIAA-2004-4465, Presented at the 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Albany, New York, Aug. 30-1, 2004. [21] Nicolai, L.M., Fundamentals of Aircraft Design, METS Inc, 6520 Kingsland Ct, San Jose, Ca 95120, 1975. [22] Sobieszczanski-Sobieski, J., James, B., Riley, M. F., Structural sizing by generalized, multilevel optimization, AIAA Journal, 25, pp. 139-145, 1987. [23] Barthelemy, J. F. M., Riley, M.F., Improved multilevel optimization approach for the design of complex engineering systems, AIAA Journal, 26, pp. 353-360, 1988 [24] Migdalas, A., Pardalos, P., Varbrand, P., eds., Multilevel Optimization: Algorithms and Applications, Kluwer, 1998. [25] Renaud, J., Gabriele, G., Improved Coordination in Non-Hierarchic System Optimization, AIAA J., Vol 31, No. 12, pp. 2367-2373, Dec. 1993. [26] Seller, R., Batill, S., Renaud, J., “Response Surface Based, Concurrent Subspace Optimzation for Multidisciplinary System Design,” 34th AIAA Aerospace Sciences Meeting, Jan. 15-18, 1996, AIAA Paper No. 96-0714. [27] Alexandrov, N.M., Lewis, R.M., “Comparative properties of collaborative optimization and other approaches to MDO,” in Engineering Design Optimization, MCB University Press, 1999. [28] R. Braun, Collaborative Optimization: An architecture for large-scale distributed design, PhD thesis, Stanford University, May 1996. Department of Aeronautics and Astronautics. [29] Shankar, J., Ribbens, C., Haftka, R., Watson, L., “Computational study of nonhierarchical decomposition algorithm,” Computational Optimization and Applications, 2:273–293, 1993.

Documents

COLLECTIVES AND COMPLEX SYSTEM DESIGNaero.stanford.edu/Reports/VKI_Collectives_Kroo_A.pdf · COLLECTIVES AND COMPLEX SYSTEM DESIGN ... the application of optimization in the design