CONTROL THEORY

IEEE Press445 Hoes Lane, P.O. Box 1331

Piscataway, NJ 08855-1331

IEEE Press Editorial BoardRobert J. Herrick, Editor in Chief

M.AkayJ. B. AndersonP. M. AndersonJ. E. Brewer

M.EdenM. E. El-HawaryR. F. HoytS. V. KartalopoulosD. Kirk

M.S.NewmanM. PadgettW.D.ReeveG. Zobrist

Kenneth Moore, DirectorofIEEE PressCatherine Faduska, SeniorAcquisitions Editor

Robert H. Bedford, AssistantAcquisitions EditorAnthony VenGraitis, ProjectEditor

Marilyn G. Catis, Marketing Manager

IEEE Control Systems Society, SponsorCSS Liaison to IEEE Press, Bruce M. Krogh

Cover design: William T. Donnelly, WT Design

Books of Related Interest from IEEE Press

PERSPECTWES IN CONTROL ENGINEERING: Technologies, Applications, and New DirectionsEdited by Tariq Samad2001 Hardcover 536 pp IEEE Order No. PC5798 ISBN 0-7803-5356-0

PHYSIOLOGICAL CONTROL SYSTEMS: Analysis,Simulation, and EstimationA volume in the IEEE Press Book series on Biomedical EngineeringMichael C. K. Khoo2000 Hardcover 344 pp IEEE Order No. PC5680 ISBN 0-7803-3408-6

THE CONTROL HANDBOOKA CRC Handbook published in cooperation with IEEE PressEdited by William S. Levine1996 Hardcover 1566 pp IEEE Order No. PC5649 ISBN 0-8493-8570-9

ROBUSTVISIONFOR VISION-BASED CONTROL OF MOTIONA volume in the SPIE/IEEE Series on Imaging Science & EngineeringEdited by Markus Vincze and Gregory D. Hager2000 Hardcover 264 pp IEEE Order No. PC5403 ISBN 0-7803-5378-1

CONTROL THEORY

Twenty-Five Seminal Papers (1932-1981)

Edited by

Tamer BasarUniversity ofIllinoisat Urbana-Champaign

Editorial Board

Brian D. O. AndersonKarl J. AstromJohn Baillieul

TamerBasar (Chair)Bruce A. FrancisAlberto Isidori

Petar V. KokotovicHuibert Kwakemaak

WilliamJ. LevineLennart Ljung

David Q. MayneJan C. Willems

IEEE Control Systems Society,Sponsor

A SelectedReprint Volume

IEEEPRESS

The Institute of Electrical and ElectronicsEngineers, Inc., New York

This book and other books may be purchased at a discountfrom the publisher when ordered in bulk quantities. Contact:

IEEE Press MarketingAttn: Special Sales

445 Hoes Lane, P.O. Box 1331Piscataway, NJ 08855-1331

Fax: +1 732 981 9334

For more information on IEEE Press products, visit theIEEE Online Catalog and Store: http://www.ieee.org/store.

© 2001 by the Institute of Electrical and Electronics Engineers, Inc.,3 Park Avenue, 17th Floor, New York, NY 10016-5997.

All rights reserved. No part of this book may be reproduced in anyform,nor may it be storedin a retrieval systemor transmitted in anyform,

withoutwrittenpermission from the publisher.

Printed in the United States of America.

10 9 8 7 6 5 4 3 2 1

ISBN 0-7803-6021-4

IEEE Order No. PC5870

Library of Congress Cataloging-in-Publication Data

Control theory: twenty-five seminal papers (1931-1981) / edited by Tamer Basar,p. em.

"IEEE Control Systems Society, sponsor.""A selected reprint volume."ISBN 0-7803-6021-41. Automatic control. 2. Control theory. I. Basar,Tamer. II. IEEE Control Systems

Society.

TJ213.7.C662000629.8--dc21

00-058171CIP

Contents

Preface vii

Regeneration Theory 1Nyquist, H. (Bell Syst. Tech. J., Vol. 11, 1932, pp. 126-147.)

Stabilized Feedback Amplifiers 25Black, H. S. (Bell Syst. Tech. J., Vol. 13, 1934, pp. 1-18.)

Relations Between Attenuation and Phase in Feedback Amplifier Design 45Bode, H. W. (Bell Syst. Tech. J., Vol. 19,1940, pp. 421-454.)

The Linear Filter for a Single Time Series 81Wiener, N. (Chapter III from Extrapolation, Interpolation, and Smoothing ofStationary Time Series,

The M.I.T. Press, Cambridge, MA, 1949, pp. 81-103.)

Control System Synthesis by Root Locus Method 107Evans, W. R. (Trans. Amer. Inst. Electric.Engineers, Vol. 69, 1950, pp. 66-69.)

The Structure of Dynamic Programming Processes 113Bellman, R. (Chapter 3 from DynamicProgramming, Princeton University Press, Princeton, NJ,

1957, pp. 81-89.)

Optimal Regulation Processes 125Pontryagin, L. S. (Uspekhi Mat. Nauk, USSR, Vol. 14,1959, pp. 3-20. (English translation: Amer.

Math. Society Trans., Series 2, Vol. 18, 1961, pp. 321-339.»

Contributions to the Theory of Optimal Control 147Kalman, R. E. (Bol. Soc. Mat. Mexicana, Vol. 5,1960, pp. 102-119.)

A NewApproach to Linear Filtering and Prediction Problems 167Kalman, R. E. [Trans. ASME (J. Basic Engineering), Vol. 82D, March 1960, pp. 35-45.]

Dual Control Theory, Parts I and II 181Feldbaum, A. A. [Automation and Remote Control, Vol. 21, April 1961, pp. 874-880, and May 1961,

pp. 1033-1039. (Russian originals dated September 1960, pp. 1240-1249, and November 1960,pp. 1453-1464.)]

Absolute Stability of Nonlinear Systems of Automatic Control 197Popov, V. M. [Automation and Remote Control, Vol. 22, February 1962, pp. 857-875. (Russian original

dated August 1961, pp. 961-979.)]

A Steepest-AscentMethod for SolvingOptimum Programming Problems 219Bryson, A. E., and Denham, W. F. [Trans. ASME (J. Appl. Mechanics), June 1962, pp. 247-257.]

v

vi CONTENTS

The Solution of Certain Matrix Inequalities in Automatic Control Theory 233Yakubovich, V. A. (DANDokladyAkademiiNauk SSSR),Vol. 143, 1962, pp. 1304-1307. (English

translation: SovietMathematics (by AmericanMath. Society), 1962, pp. 620-623.)

Mathematical Description of Linear Dynamical Systems 239Kalman, R. E. (SIAMJ. Control, Vol. 1, 1963, pp. 152-192.)

On the Input-Output Stability of Time-Varying Nonlinear Feedback Systems-Part I: Conditions derived usingconcepts of loop gain, conicity, and positivity; Part II: Conditions involving circles in the frequency planeand sector nonlinearities 283

Zames, G. (IEEETrans. Automat. Contr., Vol. AC-ll, 1966, pp. 228-238 and 465-476.)

An Invariance Principle in the Theory of Stability 309Lasalle, J. P. (in Differential Equations and Dynamical Systems, J. Hale and J. P. LaSalle, Eds.,

Academic Press, New York, 1967, pp. 277-286.)

Decoupling and Pole Assignment in Linear Multivariable Systems: A Geometric Approach 321Wonham, W. M., and Morse, A. S. (SIAMJ. Control, Vol. 8,1970, pp. 1-18.)

System Theory on Group Manifolds and Coset Spaces 341Brockett, R. W. (SIAMJ. Control, Vol. 10, 1972, pp. 265-284.)

Controllability of Nonlinear Systems 363Sussmann, H. J., and Jurdjevic, V. (J. Diff.Eqns., Vol. 12, 1972, pp. 95-116.)

Dissipative Dynamical Systems-Part I: General Theory 389Willems, J. C. (Arch. Ratl. Mech. and Analysis,Vol. 45, 1972, pp. 321-351.)

On Self-Tuning Regulators 423Astrom, K. J., and Wittenmark, B. (Automatica, Vol. 9,1973, pp. 185-199.)

Nonlinear Controllability and Observability 441Hermann, R., and Krener, A. J. (IEEETrans. Automat. Contr., Vol. AC-22, 1977, pp. 728-740.)

Analysis of Recursive Stochastic Algorithms 457Ljung, L. (IEEETrans. Automat. Contr., Vol. AC-22, 1977, pp. 551-575.)

Discrete Time Multivariable Adaptive Control 485Goodwin, G. C., Ramadge, P. J., and Caines, P. E. (IEEETrans. Automat. Contr., Vol. AC-25, 1980,

pp.449-456.)

Feedback and Optimal Sensitivity: Model Reference Transformations, Multiplicative Seminorms,and Approximate Inverses 495

Zames, G. (IEEETrans. Automat. Contr., Vol. AC-26, 1981, pp. 301-320.)

Index 517

About the Editor 523

Preface

CONTROL is in one sense a fairly young discipline.Even thoughit would be possible to push its historical origins back by abouttwo millennia to the days of the Babylonians, in modem termsthe real creation of the field has been in the twentieth century.It is in this century that CONTROL became a scientificdiscipline,with an intellectual core shaped by revolutionary ideas, novelconcepts, and a wealth of analytical and computational tools.As a young and intellectuallystimulatingdiscipline, it attractedsomeof thebrightestmindsto itsranksand,withits theorydrivenby real applications, it provided versatile tools for generationsof practicing engineers.

Prior to the twentieth century, there were of course also sev-eral key contributions. Perhaps the first time the studyof controlsystemsattracted serious scientificattentionwas towardthe endof the eighteenth century, following James Watts's inventionof the governor in 1788, which was designed to regulate thespeed of the rotary steam engine. A related work on governorsby Huygens! actuallypredated that of Wattsby about a century;Huygensinventedthecentrifugalgovernoras ameansof regulat-ing a clock, which was adapted for windmills and water wheelsin the Netherlandsas early as the seventeenthcentury.Feedbackplayed an important role in all these inventions,and soon it wasrealized and widely acknowledged that it is a concept that liesat the foundation of any successful control design. However, tomake effective use of feedback, there was a need for a carefulmathematicalstudy of its impact on control design.James ClerkMaxwellwas the first to realize this need and to respond to it bydevelopingin his now famous paperv'consideredby many to bethe starting point of the scientificapproach to control research,mathematicalmodelsforvariousgovernormechanismsbasedonlinear differentialequations. He worked out in his paper a com-plete theoryof stabilityfor constantcoefficientlineardifferentialequations up to fourth order, and obtained some conditions forstabilityof fifth-order systems.Around the same time, and inde-pendently of Maxwell,a Russianengineer,Vyshnegradskii, hadrecognized the importanceof control in industrial applications,

IC. Huygens, "Horologii oscillatorii," Part 5, Paris, 1673.2J.C. Maxwell, "Ongovemors," Proc.RoyalSoc.London,16, 1868,pp. 270-

283.

vii

and the need for developing a sound theory.' Where Maxwellleft off was then picked up by Edward John Routh," and inde-pendentlyby Adolf Hurwitz.l who came up with what is knowntodayas theRouth-Hurwitz stabilitycriteria,solvingcompletelythe problem of stability of constant coefficient linear differen-tial equations of any finite order. At about the same time, andas the nineteenth century was coming to a close, another trend-settingdevelopmenttookplace, again in the areaof stability, butthis time for nonlinear dynamical systems. Motivatedby prob-lems that arise in astronomy in connection with the motion ofthe planets, a topic studied earlier by Henry Poincare," amongothers,AleksandrMikhailovichLyapunovdevelopedin his doc-toral thesis in Russia a new approach for testing the stability ofthe equilibrium of a system described by nonlinear ordinarydifferential equations, known today as the Second Method ofLyapunov.

Hence, there wasquite a bit of accumulatedactivityin controlat the beginning of the twentieth century. But what this century,and particularly its secondhalf, deliveredwas somethingdiffer-ent in terms of both content and sheer volume of diverse con-tributions.The incessant growth caused by an explosionof newfresh ideas, and drivenby numerous applicationsfrom differentdomains, brought this activity to unprecedented levels. As weare coming to the close of this century,we thought that it wouldbe useful to reflectback and ask the questions: What have beenthe major results of this century in control? What have been thegreatest hits in control? How has control theory evolved sincethe times of Maxwell, Routh, Hurwitz, and Lyapunov (amongothers)?There is of course no unique way of answeringall thesequestions, but one possible way is to collect under one cover,

3J. Vyshnegradskii, "Sur la theorie generale des regulateurs (On the generaltheory of control)," Compt. Rend.Acad. Sci. Paris, 83,1876, pp. 318-321.

4E.J. Routh, A Treatise on theStabilityofa GivenStateofMotion,Macmillan(London), 1877.

5A. Hurwitz, "Uber the Bedingungen unter welschen eine Gleichung nurWurzeln mit negativen reelen Teilen besitzt (On the conditions under which anequation has only roots with negative real parts)," Mathematische Annalen,46,1895, pp. 273-284.

6H. Poincare, Les MethodesNouvellesde la Mechanique Celeste (The NewMethods of the Cellestial Mechanics), Vol. 1, Gauthier-Villars (Paris), 1892.

viii PREFACE

and as a representative of the major research developments andaccomplishments in control in this century, some key papers thathave made major impact on the field, along with some introduc-tory material for each.

These considerations have led to the present volume, whichcontains twenty-five carefully selected seminal papers coveringthe period 1932-1981, and begins with Harry Nyquist's famous"Regeneration Theory" paper, which introduced the so-calledNyquist criterion (still a versatile tool for control engineers) andlaid the foundations of a frequency-domain approach to stabil-ity analysis of linear control systems. The volume ends with the1981 paper by George Zames, which marks the beginning ofthe "robustness" era in control-an era that we are leaving tothe coming generations to evaluate (perhaps by the middle of thetwenty-first century), along with other exciting developments thecontrol field has experienced for the past two decades, and willundoubtedly continue to do so in the next century.

This volume was prepared under the auspices of the IEEEControl Systems Society, by an Editorial Board consisting oftwelve members, namely:

Brian D.O. Anderson, Karl J. Astrom, John Baillieul, TamerBasar (Chair), Bruce A. Francis, Alberto Isidori, Petar ~

Kokotovic, Huibert Kwakernaak, William J. Levine, LennartLjung, David Q. Mayne, and Jan C. Willems.

Based on nominations received in response to solicitations thatappeared in the IEEE Control Systems Magazine, and in theE-LETTER on Systems, Control, and Signal Processing (basedin the Netherlands), and nominations generated by the Boardmembers, and after several rounds of voting, the Board unan-imously agreed on the selection of the twenty-five papers in-cluded in this volume. The preambles to each paper were writtenby the Board members (in some cases jointly), with the primaryauthor(s) in each case identified by his (their) initials.

The twenty-five papers included in this volume cover a broadspectrum of major developments in control theory in the twen-tieth century, but still the volume should not be viewed as pro-viding an exhaustive coverage of all topical areas of control, asthis has not been a criterion set by the Editorial Board in theirselection of the papers. The focus here has been the papersselected rather than the areas of research in control. Still, we be-lieve that the selected papers clearly outline the path which thecontrol discipline has followed during its rapid growth from the1930s through the 1980s. To help the reader in this journey, inthe preambles to individual papers we have discussed develop-ments in areas neighboring the topic of a particular paper, so asto place its contributions and impact in proper perspective, and tomaintain continuity in the flow of ideas from one topic to another.

What is the path we can trace from the chronologically or-dered papers in the volume? First comes the basic feedback the-ory, as represented by the first three papers, by Nyquist, Black,and Bode, which was the outcome of research conducted atthe Bell Laboratories in the 1930s driven by the need to de-velop electronic feedback amplifiers for long telephone lines.To this was added later, as a practical tool, the root locus methodof Evans. In the 1940s, Wiener's work on prediction, filtering,and smoothing for time series, impacted control in many ways,

fostering further developments not only in filtering theory butalso in control design. The "Sputnik effect" and the ensuingspace research propelled the development of a mathematicallyadvanced optimal control theory in the late 1950s and early1960s, with dynamic programming, maximum principle, andthe LQ regulator design (with its associated concepts of con-trollability and observability) as its centerpieces, as representedin the works of Bellman, Pontryagin, and Kalman included inthis volume. Also developed during this period were compu-tational techniques to make dynamic programming and maxi-mum principle practicable, as represented by the paper of Brysonand Denham. During the same period, efforts were intensifiedto develop an applicable stability theory of nonlinear feedbacksystems in the absolute stability framework, represented by thepapers of Popov, Yakubovich, and Zames and, as extensions andrefinements of Lyapunov concepts, in the papers by LaSalle andWillems. The need to operate in the presence of noise and otherdisturbances was also recognized in the early 1960s, as shown inpapers by Kalman and Feldbaum. Then came the establishmentof the precise relationship between input-output descriptions andstate-space representations of linear systems, with its multifoldramifications, as presented in the 1963 paper by Kalman, andthe establishment of a geometric theory for linear systems, in thepaper by Wonham and Morse, with the introduction of the novelconcept of controlled invariance, which found applications inmuch broader domains (than linear systems) later. We see inthe 1970s the emergence of a nonlinear system theory, with as-sociated richer concepts of controllability and observability, asshown in the papers by Brockett, Sussmann and Jurdjevic, andHermann and Krener. Adaptive control is another area where acomprehensive theory started emerging in the 1970s, where theimportant notion of "self-tuning" was introduced (in the paperby Astrom and Wittenmark), and methodologies were developedfor establishing the convergence of adaptive control algorithms,as well as recursive stochastic algorithms, as presented in thepapers by Goodwin, Ramadge and Caines, and Ljung. Robustcontrol is yet another topic that gained steam in the late 1970s,with the 1981 paper by Zames (the last paper in this volume)marking the beginning of some intense activity in this domain.Of course, there were many other accomplishments in controlduring this period, than those represented by the twenty-five pa-pers selected, and it is hoped that the preambles to the paperswill convey to the reader this richness in the field.

I hope that the reader will enjoy this journey, and will de-velop a real sense of the evolution of the control field duringthe fifty-year period, 1932-1981, through the milestone accom-plishments embodied in these twenty-five seminal papers andfurther discussed in the introductory material provided by ourEditorial Board. It is our hope that the volume will be a valu-able resource in the twenty-first century (and beyond) especiallyfor young control researchers and engineers, and will be instru-mental in the realization of an even more explosive century forcontrol theory and its applications.

Tamer BasarUniversity ofIllinois at Urbana-Champaign