An Introduction to Control TheoryWith Applications to Computer Science

Joseph HellersteinAnd

Sujay ParekhIBM T.J. Watson Research Center

{hellers,sujay}@us.ibm.com

2

Example 1: Liquid Level System

H

Input valve control

float

Output valve

Goal: Design the input valve control to maintain a constant height regardless of the setting of the output valve

iq

oqV

(height)

(input flow)

(output flow)

(volume)

R(resistance)

3

Example 2: Admission Control

Users

Administrator

Controller

RPCs

Sensor

Server

Referencevalue

QueueLength

Tuningcontrol

ServerLog

Goal: Design the controller to maintain a constant queue length regardless of the workload

4

Why Control Theoryn Systematic approach to analysis and design

n Transient responsen Consider sampling times, control frequencyn Taxonomy of basic controlsn Select controller based on desired characteristics

n Predict system response to some inputn Speed of response (e.g., adjust to workload

changes)n Oscillations (variability)

n Approaches to assessing stability and limit cycles

5

Example: Control & Response in an Email Server

Control(MaxUsers)

Response(queue length)

Good

Slow

Bad

Useless

6

Examples of CT in CSn Network flow controllers (TCP/IP – RED)

n C. Hollot et al. (U.Mass)

n Lotus Notes admission controln S. Parekh et al. (IBM)

n QoS in Cachingn Y. Lu et al. (U.Va)

n Apache QoS differentiationn C. Lu et al. (U.Va)

7

Outlinen Examples and Motivationn Control Theory Vocabulary and Methodologyn Modeling Dynamic Systemsn Standard Control Actionsn Transient Behavior Analysisn Advanced Topicsn Issues for Computer Systemsn Bibliography

8

Feedback Control System

PlantControllerΣ)(tu

)(tnDisturbance

)(ty)(tr

)(te

Transducer

)(tb

)()()( tbtrte −=+

–

Reference Value

9

Controller Design Methodology

Block diagram

construction

Model Ok?

Stop

Start

Transfer function formulation and

validation

Controller Design

Objective achieved?

Controller Evaluation

Y

Y

N N

System Modeling

10

Control System Goalsn Regulation

n thermostat, target service levelsn Tracking

n robot movement, adjust TCP window to network bandwidth

n Optimizationn best mix of chemicals, minimize response

times

11

System Modelsn Linear vs. non-linear (differential eqns)

n eg, n Principle of superposition

n Deterministic vs. Stochasticn Time-invariant vs. Time-varying

n Are coefficients functions of time?n Continuous-time vs. Discrete-time

n t ∈ R vs k ∈ Z

xbxbyaya 0201 +=+ &&&

12

Approaches to System Modelingn First Principles

n Based on known lawsn Physics, Queueing theory

n Difficult to do for complex systemsn Experimental (System ID)

n Statistical/data-driven modelsn Requires datan Is there a good “training set”?

13

The Complex Plane (review)Imaginary axis (j)

Real axis

jyxu +=

x

yφ

r

rφ−

jyxu −=(complex) conjugate

y−

22

1

||||

tan

yxuru

xy

u

+=≡≡

=≡∠ −φ

14

Basic Tool For Continuous Time: Laplace Transform

n Convert time-domain functions and operations into frequency-domain n f(t) → F(s) (t∈þ, s∈ÿ)n Linear differential equations (LDE) → algebraic

expression in Complex plane

n Graphical solution for key LDE characteristicsn Discrete systems use the analogous z-transform

∫∞ −==

0)()()]([ dtetfsFtf stL

15

Laplace Transforms of Common FunctionsName f(t) F(s)

Impulse

Step

Ramp

Exponential

Sine

1

s1

2

1s

as −1

22

1s+ω

1)( =tf

ttf =)(

atetf =)(

)sin()( ttf ω=

>=

=0001

)(tt

tf

16

Laplace Transform Properties

[ ] [ ]

)(lim)(lim

)(lim)0(

)()()

)(1)(

)(

)0()()(

)()()]()([

0

02121

0

2121

ssFtf-

ssFf-

sFsFdt(tt)f(tf

dttfss

sFdttfL

fssFtfdtd

L

sbFsaFtbftafL

st

s

t

t

→∞→

∞→

±=

=

=+

=−

+=

±−=

±=±

∫

∫∫

theorem valueFinal

theorem valueInitial

nConvolutio

nIntegratio

ationDifferenti

calingAddition/S

17

Insights from LaplaceTransformsn What the Laplace Transform says about f(t)

n Value of f(0)n Initial value theorem

n Does f(t) converge to a finite value?n Poles of F(s)

n Does f(t) oscillate?n Poles of F(s)

n Value of f(t) at steady state (if it converges)n Limiting value of F(s) as s->0

18

Transfer Functionn Definition

n H(s) = Y(s) / X(s)n Relates the output of a linear system

(or component) to its inputn Describes how a linear system responds

to an impulsen All linear operations allowed

n Scaling, addition, multiplication

H(s)X(s) Y(s)

19

Block Diagramsn Pictorially expresses flows and relationships

between elements in systemn Blocks may recursively be systemsn Rules

n Cascaded (non-loading) elements: convolutionn Summation and difference elements

n Can simplify

20

Block Diagram of System

PlantControllerΣ)(sU

)(sN

Disturbance

)(sY

)(sR)(sE

Transducer

)(sB

+

–

Σ

)(1 sG )(2 sG

)(sH

Reference Value

21

Combining Blocks

Combined BlockΣ)(sY

)(sR)(sE

Transducer

)(sB

+

–

)(sH

)())()(( 21 sGsNsG •+

Reference Value

22

Block Diagram of Access Control

M(z)

G(z) N(z) S(z)

Controller Notes Server

Sensor

R(z) +E(z)

U(z)

-

Q(z)Σ

Users

Controller Sensor

ServerServerLog

23

Key Transfer Functions

)()()()(

)()(

)()(

21 sGsGsEsU

sUsY

sEsY

== :eedforwardF

)()()()()(

21 sHsGsGsEsB

= :Loop-penO)()()(1

)()()()(

:21

21

sHsGsGsGsG

sRsY

+=Feedback

PlantControllerΣ)(sU

)(sY

)(sR)(sE

Transducer)(sB

+

–

)(1 sG )(2 sG

)(sH

Reference

24

Rational Laplace Transforms

m

sFsAssFsBs

bsbsbsB

asasasA

sBsA

sF

mm

nn

poles # system ofOrder complex are zeroes and Poles

(So, :Zeroes (So, :Poles

==

==∋∞==∋

+++=

+++=

=

)0*)(0*)(*)*)(0*)(*

...)(

...)(

)()(

)(

01

01

25

First Order System

Reference

)(sY)(sR

Σ )(sE

1)(sB

)(sUsT+1

1K

sTK

sTKK

sRsY

+≈

++=

11)()(

26

First Order System

Ramp, step, exponential

Ramp response

Step, exponential

Step response

ExponentialImpulse response

1 sTK

+

/1

2 TsKT

-s

KT-

sK

+

/1

Ts

K-

sK

+

No oscillations (as seen by poles)

27

Second Order System

:frequency natural Undamped

where :ratio Damping

(ie,part imaginary zero-non have poles if Oscillates

:response Impulse

JK

JKBBB

JKB

ssKBsJsK

sRsY

N

cc

NN

N

=

==

<−

++=

++=

ω

ξ

ωξωω

2

)04

2)()(

2

22

2

2

28

Second Order System: Parameters

noscillatio the offrequency the gives frequency natural undamped of tionInterpreta

0)Im0,(Re Overdamped 1 Im) (Re dUnderdampe

0)Im 0,(Re noscillatio Undamped ratio damping of tionInterpreta

Nω

ξξ

ξ

=≠≤≠≠<<

≠==

:0:10

:0

29

Transient Response Characteristics

statesteady of % specified within stays time Settling :

reached is valuepeak whichat Time :

valuestatesteady reachfirst untildelay time Rise :

valuestatesteady of 50% reach untilDelay :

=

=

s

p

r

d

t

t

tt

0.5 1 1.5 2 2.5 3

0.25

0.5

0.75

1

1.25

1.5

1.75

2

rt

overshoot maximum=pM

pt stdt

30

Transient Responsen Estimates the shape of the curve based

on the foregoing points on the x and y axis

n Typically applied to the following inputsn Impulsen Stepn Rampn Quadratic (Parabola)

31

Effect of pole locations

Faster Decay Faster Blowup

Oscillations(higher-freq)

Im(s)

Re(s)(e-at) (eat)

32

Basic Control Actions: u(t)

:control alDifferenti

:control Integral

:control alProportion

sKsEsU

tedtd

Ktu

sK

sEsU

dtteKtu

KsEsU

teKtu

dd

it

i

pp

==

==

==

∫

)()(

)()(

)()(

)()(

)()(

)()(

0

33

Effect of Control Actionsn Proportional Action

n Adjustable gain (amplifier)

n Integral Actionn Eliminates bias (steady-state error)n Can cause oscillations

n Derivative Action (“rate control”)n Effective in transient periodsn Provides faster response (higher sensitivity)n Never used alone

34

Basic Controllers

n Proportional control is often used by itself

n Integral and differential control are typically used in combination with at least proportional controln eg, Proportional Integral (PI) controller:

+=+==

sTK

sK

KsEsU

sGi

pI

p

11

)()(

)(

35

Summary of Basic Controln Proportional control

n Multiply e(t) by a constant

n PI controln Multiply e(t) and its integral by separate constantsn Avoids bias for step

n PD controln Multiply e(t) and its derivative by separate constantsn Adjust more rapidly to changes

n PID controln Multiply e(t), its derivative and its integral by separate

constantsn Reduce bias and react quickly

36

Root-locus Analysisn Based on characteristic eqn of closed-loop

transfer functionn Plot location of roots of this eqn

n Same as poles of closed-loop transfer functionn Parameter (gain) varied from 0 to ∞

n Multiple parameters are okn Vary one-by-onen Plot a root “contour” (usually for 2-3 params)

n Quickly get approximate resultsn Range of parameters that gives desired response

37

Digital/Discrete Controln More useful for computer systemsn Time is discrete

n denoted k instead of tn Main tool is z-transform

n f(k) → F(z) , where z is complexn Analogous to Laplace transform for s-domain

n Root-locus analysis has similar flavorn Insights are slightly different

∑∞

=

−==0

)()()]([k

kzkfzFkfZ

38

z-Transforms of Common FunctionsName f(t) F(z)

Impulse

Step

Ramp

Exponential

Sine

1

1−zz

2)1( −zz

aezz

−

1)(Cos2Sin

2 +− zazaz

1)( =tf

ttf =)(

atetf =)(

)sin()( ttf ω=

F(s)

1

s1

2

1s

as −1

22

1s+ω

>=

=0001

)(tt

tf

39

Root Locus analysis of Discrete Systems

n Stability boundary: |z|=1 (Unit circle)n Settling time = distance from Originn Speed = location relative to Im axis

n Right half = slowern Left half = faster

40

Effect of discrete poles

|z|=1

Longer settling time

Re(s)

Im(s)

Unstable

Stable

Higher-frequencyresponse

Tsez = :Intuition

41

System ID for Admission Control

M(z)

G(z) N(z) S(z)

Controller Notes Server

Sensor

R(z) +E(z)

U(z)

-

Q(z)Σ

ARMA Models

Control Law

Transfer Functions

δzzzK

czdzd

azzb

zGzSzN i 11

)()()(1

10

1

0

−−+

−=Open-Loop:

δzzzK

zG

czdzd

zS

azzb

zN

i 11

)(

)(

)(

1

10

1

0

−=

−+

=

−=

)()1()(

)1()()1()(

)()1()(

101

01

teKtutu

tqdtqdtmctm

tubtqatq

i+−=

−++−=+−=

42

Root Locus Analysis of Admission Control

Predictions:•Ki small => No controller-induced oscillations•Ki large => Some oscillations•Ki v. large => unstable system (d=2)•Usable range of Ki for d=2 is small

43

Experimental Results

Control(MaxUsers)

Response(queue length)

Good

Slow

Bad

Useless

44

Advanced Control Topicsn Robust Control

n Can the system tolerate noise?n Adaptive Control

n Controller changes over time (adapts)n MIMO Control

n Multiple inputs and/or outputsn Stochastic Control

n Controller minimizes variancen Optimal Control

n Controller minimizes a cost function of error and control energyn Nonlinear systems

n Neuro-fuzzy controln Challenging to derive analytic results

45

Issues for Computer Sciencen Most systems are non-linear

n But linear approximations may don eg, fluid approximations

n First-principles modeling is difficultn Use empirical techniques

n Control objectives are differentn Optimization rather than regulation

n Multiple Controlsn State-space techniquesn Advanced non-linear techniques (eg, NNs)

46

Selected Bibliographyn Control Theory Basics

n G. Franklin, J. Powell and A. Emami-Naeini. “Feedback Control of Dynamic Systems, 3rd ed”. Addison-Wesley, 1994.

n K. Ogata. “Modern Control Engineering, 3rd ed”. Prentice-Hall, 1997.n K. Ogata. “Discrete-Time Control Systems, 2nd ed”. Prentice-Hall, 1995.

n Applications in Computer Sciencen C. Hollot et al. “Control-Theoretic Analysis of RED”. IEEE Infocom 2001 (to appear).n C. Lu, et al. “A Feedback Control Approach for Guaranteeing Relative Delays in Web

Servers”. IEEE Real-Time Technology and Applications Symposium, June 2001.n S. Parekh et al. “Using Control Theory to Achieve Service-level Objectives in

Performance Management”. Int’l Symposium on Integrated Network Management, May 2001

n Y. Lu et al. “Differentiated Caching Services: A Control-Theoretic Approach”. Int’l Conf on Distributed Computing Systems, Apr 2001

n S. Mascolo. “Classical Control Theory for Congestion Avoidance in High-speed Internet”. Proc. 38th Conference on Decision & Control, Dec 1999

n S. Keshav. “A Control-Theoretic Approach to Flow Control”. Proc. ACM SIGCOMM, Sep 1991

n D. Chiu and R. Jain. “Analysis of the Increase and Decrease Algorithms for Congestion Avoidance in Computer Networks”. Computer Networks and ISDN Systems, 17(1), Jun 1989