ARTICLE IN PRESS

Journal of Theoretical Biology 253 (2008) 824– 837

Contents lists available at ScienceDirect

Journal of Theoretical Biology

0022-51

doi:10.1

� Corr

E-m

chchang

journal homepage: www.elsevier.com/locate/yjtbi

Robust model matching control of immune systems under environmentaldisturbances: Dynamic game approach

Bor-Sen Chen a,�, Chia-Hung Chang a, Yung-Jen Chuang b

a Laboratory of Control and Systems Biology, Department of Electrical Engineering, National Tsing Hua University, Hsinchu 30013, Taiwan, ROCb Institute of Bioinformatics and Structural Biology, National Tsing Hua University, Hsinchu 30013, Taiwan, ROC

a r t i c l e i n f o

Article history:

Received 2 January 2008

Received in revised form

15 April 2008

Accepted 17 April 2008Available online 29 April 2008

Keywords:

Pathogen–host interaction network

Desired immune response

Minimax problem

Fuzzy approximation

LMI

93/$ - see front matter & 2008 Elsevier Ltd. A

016/j.jtbi.2008.04.024

esponding author. Tel.: +886 3 573 1155; fax:

ail addresses: bschen@ee.nthu.edu.tw (B.-S. C

70756@gmail.com (C.-H. Chang).

a b s t r a c t

A robust model matching control of immune response is proposed for therapeutic enhancement to

match a prescribed immune response under uncertain initial states and environmental disturbances,

including continuous intrusion of exogenous pathogens. The worst-case effect of all possible

environmental disturbances and uncertain initial states on the matching for a desired immune

response is minimized for the enhanced immune system, i.e. a robust control is designed to track a

prescribed immune model response from the minimax matching perspective. This minimax matching

problem could herein be transformed to an equivalent dynamic game problem. The exogenous

pathogens and environmental disturbances are considered as a player to maximize (worsen) the

matching error when the therapeutic control agents are considered as another player to minimize the

matching error. Since the innate immune system is highly nonlinear, it is not easy to solve the robust

model matching control problem by the nonlinear dynamic game method directly. A fuzzy model is

proposed to interpolate several linearized immune systems at different operating points to approximate

the innate immune system via smooth fuzzy membership functions. With the help of fuzzy

approximation method, the minimax matching control problem of immune systems could be easily

solved by the proposed fuzzy dynamic game method via the linear matrix inequality (LMI) technique

with the help of Robust Control Toolbox in Matlab. Finally, in silico examples are given to illustrate the

design procedure and to confirm the efficiency and efficacy of the proposed method.

& 2008 Elsevier Ltd. All rights reserved.

1. Introduction

A dynamic response of the immune system, which includesinnate immune system and adaptive immune system, isinduced by infectious microbes or environmental disturbances.The innate immune system provides a tactical response, signalingthe presence of ‘non-self’ organisms and activating B cells toproduce antibodies to bind to the intruders’ epitopic sites. Theantibodies identify targets for scavenging cells that engulf andconsume the microbes, reducing them to non-functioning units(Stengel et al., 2002b). The antibodies can also stimulate theproduction of cytokines, complement factors and acute-phaseresponse proteins that either damage an intruder’s plasmamembrane directly or trigger the second phase of immuneresponse. The innate immune system protects against manyextracellular bacteria or free viruses found in blood plasma,lymph, tissue fluid, or interstitial space between cells, but it

ll rights reserved.

+886 3 5715971.

hen),

cannot clean out microbes that burrow into cells, such as viruses,intracellular bacteria, and protozoa (Lydyard et al., 2000; Stengelet al., 2002b; Janeway, 2005).

Activated by the innate immune response, the adaptiveimmune system could provide strategic response to invadingmicrobe and yield protective cells. These protective cells couldremember specific antigens and produce antibodies to counter theantigens, and seek for epitopes of antigens on the surfaces ofinfected cells. It is found that adaptive immune mechanismsdepend on the actions of B- and T-lymphocytes that becomededicated to a single antibody type through clonal selection.Meanwhile, killer T-cells (or cytotoxic T-lymphocytes) bind toinfected cells and kill them by initiating programmed cell death(apoptosis). In addition, helper T-cells assist naive B-cells inmaturing into plasma cells that produce the needed antibodytype. Finally, immune cells with narrowly focused memory aregenerated, ready to respond rapidly if invading microbes with thesame antigen epitopes are encountered again. Elements of theinnate and adaptive immune systems are shared, and responsemechanisms are coupled, even though distinctive modes ofoperation can be identified (Lydyard et al., 2000; Stengel et al.,2002b; Janeway, 2005).

ARTICLE IN PRESS

B.-S. Chen et al. / Journal of Theoretical Biology 253 (2008) 824–837 825

For clinical treatment of infection, the current therapeuticmethods focus on killing the invading microbes, neutralizing theirresponse, and providing palliative or healing care to affectedorgans of the body. Few biological or chemical agents haveadverse side effects; for example, an agent that kills a virus mayalso damage healthy ‘self’ cells. A critical function of drugdiscovery and development is hence to identify new compoundsthat have maximum intended efficacy with minimal side effectson the general population. These examples include antibiotics asmicrobe killers; interferons as microbe neutralizers; interleukins,antigens from killed (i.e. non-toxic) pathogens, and pre-formedand monoclonal antibodies as immunity enhancers; and anti-inflammatory and anti-histamine compounds as palliative drugs(Stengel et al., 2002b).

Recently, there are many models of immune response toinfection (Asachenkov, 1994; Rundell et al., 1995; Perelson andWeisbuch, 1997; Nowak and May, 2000) with special emphasis onthe human-immunodeficiency virus (Perelson et al., 1993, 1996;Nowak et al., 1995; Stafford et al., 2000). Some papers havediscussed immune defense models with moving target strategy(Adler and Karban, 1994). Wiener (1948) and Bellman (1983)appreciated and anticipated the application of mathematicalanalysis to treatment in a broad sense, and Swan (1981) surveysearly optimal control applications to biomedical problems.Notably, Kirschner et al. (1997) offers an optimal control approachto HIV treatment, and intuitive control approaches are presented(De Boer and Boucher, 1996; Bonhoeffer et al., 1997; Wein et al.,1998; Wodarz and Nowak, 1999, 2000).

The dynamics of drug response (pharmacokinetics) have beenmodeled in several works (Robinson, 1986; Van rossum et al.,1986) and control theory is applied to drug delivery in otherstudies (Bell and Katusiime, 1980; Carson et al., 1985; Chizeck andKatona, 1985; Jelliffe, 1986; Schumitzky, 1986; Kwong et al., 1995;Polycarpou and Conway, 1995; Parker et al., 1996; Gentilini et al.,2001). Recently, Stengel et al. (2002a) presented a simple modelfor the response of the innate immune system to infection andtherapeutic intervention, reviewed the prior method and resultsof optimization, and introduced a significant extension to theoptimal control of enhancing immune response by solving a two-point boundary-value problem via an iterative method. Theirresults show not only the progression from an initially life-threatening state to a controlled or cured condition but also theoptimal history of therapeutic agents that produces that condi-tion. In their study, the therapeutic method is extended by addinglinear-optimal feedback control to the nominal optimal solution.However, the performance of quadratic optimal control forimmune systems may be decayed by the continuous exogenouspathogens input, which is considered as an environmentaldisturbance of the immune system. Furthermore, some over-shoots may occur in the optimal control process and may lead toorgan failure because the quadratic optimal control only mini-mizes a quadratic cost function that is only the integration ofsquares of states and allows the existence of overshoot (Zhouet al., 1996). A series dynamic optimization method is thereforeproposed to design the optimal schedule for host defense,immune memory and post-infection pathogen levels in mammals(Shudo and Iwasa, 2001, 2002, 2004; Shudo et al., 2003).

In this study, a robust model matching control of immuneresponse is proposed for therapeutic enhancement to match adesired immune response under uncertain exogenous pathogensinput, environmental disturbances and uncertain initial states.Because of the uncertainties of these factors mentioned above, inorder to attenuate their detrimental effects, their worst-caseeffects should be considered in the matching control procedurefrom the robust design perspective. The worst-case effect of allpossible uncertain factors on the matching error to a desired

immune response is minimized for the enhanced immunesystems, i.e. the proposed robust model matching control isdesigned from the minimax matching perspective. This minimaxmatching could be transformed to an equivalent dynamic gameproblem (Basar and Olsder, 1999). The exogenous pathogens inputis considered as a player to maximize (worsen) the matchingerror, while the therapeutic control agent is considered as anotherplayer to minimize the matching error. Since the innate immunesystem is highly nonlinear, it is not easy to solve the robust modelmatching control problem by the nonlinear dynamic gamemethod directly. Recently, fuzzy systems have been employed toefficiently approximate nonlinear dynamic systems to solve thenonlinear control problem (Chen et al., 1999, 2000; Lian et al.,2001; Li et al., 2004). A fuzzy model is proposed to interpolateseveral linearized immune systems at different operating pointsto approximate the innate immune system via smooth fuzzymembership functions. Then, with the help of fuzzy approxima-tion method, a fuzzy dynamic game scheme is developed so thatthe minimax matching control of immune systems could be easilysolved by the linear dynamic game method, which can besubsequently solved by a constrained optimization scheme viathe linear matrix inequality (LMI) technique (Boyd, 1994) with thehelp of Robust Control Toolbox in Matlab. Because the fuzzydynamic model can approximate any nonlinear dynamic system,the proposed model matching method via fuzzy game theory canbe applied to the robust control design of any model of immunesystem that can be Takagi–Sugeno (T–S) fuzzy interpolated.Finally, the computational simulation examples considering theside effect of agents are given to illustrate the design procedureand to confirm the efficiency and efficacy of the proposedminimax match control method for immune systems.

2. Model of immune response

For the principal goals to study the general course of a diseaseand to clarify some observational results, a simple four-nonlinear,ordinary differential equation for the dynamic model of infectiousdisease is introduced as the following equations to describe ratesof change of pathogen, immune cell and antibody concentrationsand of an indicator of organic health (Asachenkov, 1994; Stengel etal., 2002a). A more general dynamic model will be given next insequel:

_x1 ¼ ða11 � a12x3Þx1 þ b1u1 þw1,

_x2 ¼ a21ðx4Þa22x1x3 � a23ðx2 � x�2Þ þ b2u2 þw2,

_x3 ¼ a31x2 � ða32 þ a33x1Þx3 þ b3u3 þw3,

_x4 ¼ a41x1 � a42x4 þ b4u4 þw4,

a21ðx4Þ ¼cosðpx4Þ; 0px4p1=2;

0; 1=2px4;

((1)

where x1 denotes the concentration of a pathogen that expresses aspecific foreign antigen, x2 denotes the concentration of immunecells that are specific to the foreign antigen, x3 denotes theconcentration of antibodies that bind to the foreign antigen, x4

denotes the characteristic of a damaged organ (x4 ¼ 0: healthy,x4X1: dead). The combined therapeutic control agents and theexogenous inputs are described as follows: u1 denotes thepathogen killer’s agent, u2 denotes the immune cell enhancer, u3

denotes the antibody enhancer, u4 denotes the organ healingfactor (or health enhancer), and w1 denotes the rate of continuingintroduction of exogenous pathogens. w2–w4 denote the environ-mental disturbances or unmodeled errors and residues. a21(x4) is anonlinear function that describes the mediation of immune cellgeneration by the damaged cell organ. And if there is no antigen,then the immune cell maintains the steady equilibrium value of

ARTICLE IN PRESS

B.-S. Chen et al. / Journal of Theoretical Biology 253 (2008) 824–837826

x�2. The parameters have been chosen to produce a system thatrecovers naturally from the pathogen infections (without treat-ment) as a function of initial conditions during a period of times.For the benchmark example in Eq. (1), both parameters and timeunits are abstractions, as no specific disease is addressed. Thestate and control are always positive because concentrationscannot go below zero, and organ death is indicated when x4(t)X1.The structural relationship of system variables in Eq. (1) isillustrated in Fig. 1. Organ health mediates immune cell produc-tion, inferring a relationship between immune response andfitness of the individual. Antibodies bind to the attacking antigens,thereby killing pathogenic microbes directly, activating comple-ment proteins, or triggering an attack by phagocytic cells, e.g.macrophages and neutrophils. Each element of the state is subjectto independent control, and new microbes may continue to enterthe system.

Several typical uncontrolled responses to increasing levels ofinitial pathogen concentration under sub-clinical, clinical, chronic,and lethal conditions are shown in Fig. 2 (Stengel et al., 2002a). Ingeneral, the sub-clinical response would not require medicalexamination, while the clinical case warrants medical consulta-tion but is self-healing without intervention. Pathogen concentra-tion stabilizes at non-zero values in the chronic case, which ischaracterized by permanently degraded organ health, and itdiverges in the lethal case and expire the organ. The ‘lethal’simulation of Fig. 2 is allowed to continue to past the point atwhich x4 exceeds 1 (Stengel et al., 2002b). Finally, a more generaldisease dynamic model could be represented as

_xðtÞ ¼ f ðxðtÞÞ þ BuðtÞ þ DwðtÞ; xð0Þ ¼ x0, (2)

where xðtÞ 2 Rn�1 is the state vector, uðtÞ 2 Rm�1 is the controlagents input, wðtÞ 2 Rn�1 includes exogenous pathogens andenvironmental disturbances or uncertainty vector. f ðxðtÞÞ denotesall possible nonlinear interactions in the immune system.

3. Robust therapeutic control of immune response

The optimal control is to specify u(t) such that the followingcost function is minimized (Stengel et al., 2002a):

J ¼1

2xTðtf ÞPxðtf Þ þ

Z tf

0½xTðtÞQxðtÞ þ uTðtÞRuðtÞ�dt

� �, (3)

Fig. 1. Innate and enhanced immune response to a pathogenic attac

where P, Q, and R are weighting matrices to be specifiedby designer. Because the quadratic control is only to minimize J

in Eq. (3), i.e. the integration of xTðtÞQxðtÞ þ uTðtÞRuðtÞ to beminimized, a control leading to large overshoot of xðtÞ but withsmall integration of xTðtÞQxðtÞ may be specified in the quadraticoptimal control design (Zhou et al., 1996). This therapeutic controlwill lead to organ failure because x4(t)X1. Furthermore, the costfunction does not include exogenous pathogens and environ-mental disturbances w(t), which may degrade the performance ofthe quadratic optimal control. Therefore, it is more appealing toprescribe a desired time response of the disease dynamic in Eq. (2)beforehand. Next, we design therapeutic control agents u(t) tooptimally track the desired time response and at the same timethe influence of exogenous pathogens and environmental dis-turbances w(t) on the tracking should be eliminated as much aspossible.

Consider a reference model of immune system with a desiredtime response prescribed as follows:

_xrðtÞ ¼ ArxrðtÞ þ rðtÞ, (4)

where xrðtÞ 2 Rn�1 is the reference state vector, Ar 2 Rn�n is aspecific asymptotically stable matrix, and r(t) is a desiredreference signal. It is assumed that xrðtÞ; 8t40 representsa desired immune response for Eq. (2) to follow, i.e. thetherapeutic control is to specify u(t) such that the trackingerror ~xðtÞ ¼ xðtÞ � xrðtÞ must be as small as possible under theinfluence of uncertain exogenous pathogens and environ-mental disturbances w(t). Since the exogenous pathogensand environmental disturbances w(t) and the initial state x(0)are uncertain and reference signal r(t) could be arbitrarilyassigned, the robust model matching control design shouldbe specified so that the worst-case effect of three uncertaintiesw(t), x(0), and r(t) on the tracking error could be minimizedand set below a prescribed value r2, i.e. both the minimaxmatching and robustness against uncertainties w(t), x(0), and r(t)should be achieved simultaneously (Boyd, 1994; Basar and Olsder,1999):

minuðtÞ

maxwðtÞ;rðtÞ

R tf

0 ð~xTðtÞQ ~xðtÞ þ uTðtÞRuðtÞÞdtR tf

0 ðwTðtÞwðtÞ þ rTðtÞrðtÞÞdt þ ~xT

ð0Þ~xð0Þpr2,

8~xð0Þ, (5)

k under exogenous pathogens and environmental disturbances.

ARTICLE IN PRESS

Fig. 2. Native immune responses to pathogens which are under sub-clinical, clinical, chronic, and lethal conditions (Stengel et al., 2002a).

B.-S. Chen et al. / Journal of Theoretical Biology 253 (2008) 824–837 827

where the weighting matrices Q and R are assumed diagonal asfollows:

Q ¼

q11 0 0 0

0 q22 0 0

0 0 q33 0

0 0 0 q44

266664

377775; R ¼

r11 0 0 0

0 r22 0 0

0 0 r33 0

0 0 0 r44

266664

377775.

The diagonal element qii of Q denotes the punishment on thecorresponding tracking error and the diagonal element rii of R

denotes the relative therapeutic cost. Since the worst-case effectof w(t), r(t), and uncertain initial state x(0) on tracking error ~xðtÞand control u(t) is minimized from the energy point of view, theminimax problem of Eq. (5) is suitable for the minimax matchingproblem under unknown initial x(0), uncertain environmentaldisturbances w(t) and changeable reference r(t), which are alwaysmet in practical design cases. Because it is not easy to solvethe Nash dynamic game problem in Eq. (5) subject to Eqs. (2) and(4) directly, we provide an upper bound r2 of the minimaxproblem.

Remark 1. Actually, the design idea is the same as the modeladaptive control (MRAC) (Astrom and Wittenmark, 1995). Thedesired time response in Eq. (4) is the model reference in Astromand Wittenmark. The difficult of the model reference controldesign of immune system is that all the immune systems arenonlinear and external disturbances are uncertain. Therefore,the minimax game theory in Eq. (5) and fuzzy interpolationmethod are employed to simplify the design procedure of thenonlinear MRAC design problem of immune systems in the nextapproach.

We will first solve the above sub-minimax problem and thendecrease the upper bound r2 as small as possible to get the realminimax problem. Since the denominator in Eq. (5) is indepen-dent of u(t) and is not zero, Eq. (5) is equivalent to (Boyd, 1994;

Basar and Olsder, 1999):

minuðtÞ

maxwðtÞ;rðtÞ

Z tf

0ð~xTðtÞQ ~xðtÞ þ uTðtÞRuðtÞ � r2wTðtÞwðtÞ

� r2rTðtÞrðtÞÞdtpr2 ~xTð0Þ~xð0Þ; 8~xð0Þ. (6)

Let us denote

minuðtÞ

maxwðtÞ;rðtÞ

JðuðtÞ;wðtÞ; rðtÞÞ

¼ minuðtÞ

maxwðtÞ;rðtÞ

Z tf

0ð~xTðtÞQ ~xðtÞ þ uTðtÞRuðtÞ

� r2wTðtÞwðtÞ � r2rTðtÞrðtÞÞdt.

From the above analysis, the dynamic game problem in Eq. (5) or(6) is equivalent to finding the worst-case disturbance wnðtÞ

and reference signal rnðtÞ which maximize JðuðtÞ;wðtÞ; rðtÞÞ andthen a minimax control unðtÞ which minimizes JðuðtÞ;wnðtÞ; rnðtÞÞ

such that the minimax value JðunðtÞ;wnðtÞ; rnðtÞÞ is less thanr2 ~xð0ÞT ~xð0Þ, i.e.

JðunðtÞ;wnðtÞ; rnðtÞÞ ¼ minuðtÞ

JðuðtÞ;wnðtÞ; rnðtÞÞ

¼ minuðtÞ

maxwðtÞ;rðtÞ

JðuðtÞ;wðtÞ; rðtÞÞ

pr2 ~xTð0Þ~xð0Þ; 8~xð0Þ. (7)

Hence, if there exist unðtÞ, wnðtÞ, and rnðtÞ such that minimaxmatching problem in Eq. (7) is solved, then they can satisfy therobust model matching performance in Eq. (5) as well. Therefore,the first step of robust matching control design of therapeuticagents for immune systems is to solve the following dynamicgame problem:

minuðtÞ

maxwðtÞ;rðtÞ

JðuðtÞ;wðtÞ; rðtÞÞ (8)

subject to the disease dynamic model in Eq. (2) and the desiredreference model in Eq. (4). After that, the next step is to check

ARTICLE IN PRESS

B.-S. Chen et al. / Journal of Theoretical Biology 253 (2008) 824–837828

whether the condition Jðu�ðtÞ;w�ðtÞ; r�ðtÞÞpr2 ~xTð0Þ~xð0Þ is satisfied

or not for any ~xð0Þ.In general, it is not easy to solve the minimax matching

problem directly; it should be transformed to an equivalentminimax regulation problem. Let us denote

FðxðtÞÞ ¼f ðxðtÞÞ

ArxrðtÞ

" #; xðtÞ ¼

xðtÞ

xrðtÞ

" #2 R2n�1,

uðtÞ 2 Rm�1 and vðtÞ ¼wðtÞ

rðtÞ

" #2 R2n�1.

Then we can rewrite the minimax matching problem as

minuðtÞ

maxvðtÞ

JðuðtÞ; vðtÞÞ ¼minuðtÞ

maxvðtÞ

Z tf

0ðxTðtÞQ xðtÞ

þ uTðtÞRuðtÞ � r2vTðtÞvðtÞÞdt; 8xð0Þ (9)

subject to the following augmented system of Eqs. (2) and (4):

_xðtÞ ¼ FðxðtÞÞ þ BuðtÞ þ CvðtÞ, (10)

where

Q ¼Q �Q

�Q Q

" #; B ¼

B

0

� �; C ¼

D 0

0 I

� �; I ¼

I �I

�I I

� �,

in which I denotes the 4� 4 identity matrix. Then the minimaxmatching problem in Eq. (8) is equivalent to the followingminimax regulation problem of the augmented system in Eq. (10):

minuðtÞ

maxvðtÞ

JðuðtÞ; vðtÞÞ

¼ minuðtÞ

maxvðtÞ

Z tf

0ðxTðtÞQ xðtÞ þ uTðtÞRuðtÞ

� r2vTðtÞvðtÞÞdtpr2xTð0ÞIxð0Þ; 8xð0Þ (11)

subject to Eq. (10).

Theorem 1. The dynamic game problem for robust matching

control of immune response in Eq. (11) could be solved by the

following minimax matching control unðtÞ and the worst-case

disturbance vnðtÞ:

unðtÞ ¼ �1

2R�1BT qVðxðtÞÞ

qxðtÞ, (12)

vnðtÞ ¼1

2r2CT qVðxðtÞÞ

qxðtÞ, (13)

where VðxðtÞÞ40 is the positive solution of the following Hamilton–

Jacobi inequality (HJI):

qVðxðtÞÞ

qxðtÞ

� �T

FðxðtÞÞ þ xTðtÞQ xðtÞ

�1

4

qVðxðtÞÞ

qxðtÞ

� �T

BR�1BT qVðxðtÞÞ

qxðtÞ

þ1

4r2

qVðxðtÞÞ

qxðtÞ

� �T

CCT qVðxðtÞÞ

qxðtÞo0 (14)

with

Vðxð0ÞÞpr2xTð0ÞIxð0Þ; 8xð0Þ. (15)

Proof. See Appendix A.

Since r2 is the upper bound of Nash game problem in Eq. (5),based on the analysis above, the minimax matching control unðtÞ

and the worst-case disturbance vnðtÞ still need to minimize theupper bound r2 as follows:

r20 ¼ min

VðxðtÞÞ40r2 (16)

subject to Eqs. (14) and (15).

After solving a VðxðtÞÞ and r20 from the constrained optimization

in Eq. (16), we substitute this solution VðxðtÞÞ to obtain theminimax matching control unðtÞ in Eq. (12).

4. Robust model matching control of innate immune systemvia fuzzy interpolation method

Because it is very difficult to solve the nonlinear HJI in Eq. (14),no simple approach is available to solve the constrainedoptimization problem in Eq. (16) for robust model matchingcontrol of innate immune system. Recently (Takagi and Sugeno,1985; Chen et al., 1999, 2000), the fuzzy Takagi–Sugeno (T–S)model has been widely applied to approximate the nonlinearsystem via interpolating several linearized systems at differentoperating points so that the nonlinear dynamic game problemcould be transformed to a fuzzy dynamic game problem. Usingsuch approach, the HJI in Eq. (14) can be replaced by a set of LMIs.In this situation, the nonlinear dynamic game problem in Eq. (5)could be easily solved by fuzzy dynamic game method for thedesign of robust model matching control for innate immuneresponse systems.

Suppose the augmented system in Eq. (10) can be representedby the T–S fuzzy model (Takagi and Sugeno, 1985). The T–S fuzzymodel is a piecewise interpolation of several linearized modelsthrough membership functions. The fuzzy model is described byfuzzy ‘If–Then’ rules and will be employed to deal with thenonlinear dynamic game problem for robust model matchingcontrol to achieve a desired immune response under exogenouspathogens input and environmental disturbances. The ith rule offuzzy model for nonlinear system in Eq. (10) is of the followingform (Chen et al., 1999, 2000):

�

Rule i

If x1ðtÞ is Fi1 and . . . and xgðtÞ is Fig ,

then _xðtÞ ¼ AixðtÞ þ BuðtÞ þ CvðtÞ; i ¼ 1;2;3; . . . ; L, (17)

in which

Ai ¼Ai 0

0 Ar

" #; B ¼

B

0

� �; C ¼

D 0

0 I

� �,

and Fij is the fuzzy set; Ai, B, and C are known constant matrices; L

is the number of If–Then rules, g is the number of premisevariables and x1ðtÞ; x2ðtÞ; . . . ; xgðtÞ are the premise variables. Thefuzzy system is inferred as follows (Takagi and Sugeno, 1985; Chenet al., 1999, 2000):

_xðtÞ ¼

PLi¼1miðxðtÞÞ½AixðtÞ þ BuðtÞ þ CvðtÞ�PL

i¼1miðxðtÞÞ

¼XL

i¼1

hiðxðtÞÞ½AixðtÞ þ BuðtÞ þ CvðtÞ�, (18)

where miðxðtÞÞ ¼Qg

j¼1FijðxjðtÞÞ, hiðxðtÞÞ ¼ miðxðtÞÞ=PL

i¼1miðxðtÞÞ, xðtÞ ¼

½x1ðtÞ; x2ðtÞ; . . . ; xgðtÞ�, and FijðxjðtÞÞ is the grade of membership ofxj(t) in Fij.

We assume

miðxðtÞÞX0 andXL

i¼1

miðxðtÞÞ40. (19)

Therefore, we get

hiðxðtÞÞX0 andXL

i¼1

hiðxðtÞÞ ¼ 1. (20)

ARTICLE IN PRESS

B.-S. Chen et al. / Journal of Theoretical Biology 253 (2008) 824–837 829

The T–S fuzzy model in Eq. (18) is to interpolate L linear systemsto approximate the nonlinear system in Eq. (10) via the fuzzy basisfunction hi(x(t)). We specify the parameter Ai easily so thatPL

i¼1hiðxðtÞÞAixðtÞ in Eq. (18) can approximate FðxðtÞÞ in Eq. (10) bythe fuzzy identification method (Takagi and Sugeno, 1985).

After the nonlinear system in Eq. (10) is approximated as theT–S fuzzy system in Eq. (18), the nonlinear dynamic gameproblem in Eqs. (10) and (11) is replaced by solving the fuzzydynamic game problem in Eqs. (18) and (11).

Theorem 2. The minimax control and the worst-case disturbance for

the fuzzy dynamic game problem in Eq. (11) subject to Eq. (18) are

solved respectively as follows:

u�ðtÞ ¼ �R�1BTPxðtÞ and v�ðtÞ ¼1

r2CTPxðtÞ, (21)

where P is the positive definite symmetric matrix solution of the

following Riccati-like inequality:

PAi þ ATi P þ Q � PTBR�1BTP þ

1

r2PTCCTPp0,

i ¼ 1; . . . ; L,

Ppr2I. (22)

Proof. See Appendix B.

By fuzzy approximation, obviously, the HJI in Eq. (14) can beapproximated by a set of algebraic inequalities in Eq. (22).

Since r2 is the upper bound of minimax Nash game problem inEq. (5), the minimax game problem still needs to minimize r2 asfollows:

r20 ¼min

P40r2 (23)

subject to Eq. (22).In order to solve the above-constrained optimization in

Eq. (23) by the conventional LMI method, we let W ¼ P�140.Then Eq. (22) can be equivalent to

AiW þWATi þWQW � BR�1BT

þ1

r2CCTp0; i ¼ 1; . . . ; L,

or AiW þWATi þW

Q �Q

�Q Q

" #W � BR�1BT

þ1

r2CCTp0; i ¼ 1; . . . ; L,

or AiW þWATi þW

Q1=2

�Q1=2

24

35I Q1=2

�Q1=2h i

W

� BR�1BTþ

1

r2CCTp0; i ¼ 1; . . . ; L.

By the schur complements (Boyd, 1994), the constrained optimi-zation in Eqs. (22) and (23) is equivalent to the following LMI-constrained optimization:

r20 ¼ min

W40r2 (24)

subject to

AiW þWATi � BR�1BT

þ1

r2CCT W

Q1=2

�Q1=2

24

35

Q1=2�Q1=2

h iW �I

2666664

3777775p0,

i ¼ 1; . . . ; L,

r2WXI. (25)

Remark 2.

(1)

The fuzzy basis function hiðxðtÞÞ in Eqs. (18) and (20) can bereplaced by other interpolation functions, for example, cubicspline functions.

(2)

By fuzzy approximation, the HJI in Eq. (14) of nonlineardynamic game problem is replaced by a set of inequalities inEq. (22), which can be easily solved by LMI-constrainedoptimization in Eq. (25).

(3)

The constrained optimization to solve r0 and W ¼ P�1 inEqs. (24) and (25) can be easily solved by decreasing r2 untilthere exists no W40 solution in Eq. (25). After solving W andthen P ¼W�1 from the constrained optimization problem inEqs. (24) and (25), the minimax control can be obtained fromEq. (21).

(4)

The solution W40 in LMI-constrained optimization Eq. (25)can be solved by Robust Control Toolbox in Matlab efficiently.

(5)

If the conventional quadratic optimal control in Eq. (3) isconsidered (Stengel et al., 2002b), i.e. the effect of disturbanceis not considered in the design procedure, the optimaltracking control problem is equivalent to letting r2 ¼ 1 inEq. (5) (Zhou et al., 1996 ). Then the optimal control designu�ðtÞ ¼ �R�1BPxðtÞ can be solved by a common positivedefinite symmetric matrix P from Eq. (22) with r2 ¼ 1, i.e.solving a common positive definite symmetric matrix P40from the following constrained inequalities: PAi þ AT

i P þ Q �PTBR�1BTPp0; i ¼ 1; . . . ; L (Boyd, 1994). In order to solve theoptimal tracking control by LMI technique, the optimaltracking control is equivalent to solving a common W ¼ P�1

from the following constrained inequalities:

AiW þWATi þW

Q1=2

�Q1=2

24

35I Q1=2

�Q1=2h i

W

� BR�1BTp0; i ¼ 1; . . . ; L (26)

or equivalently,

AiW þWATi � BR�1BT W

Q1=2

�Q1=2

24

35

Q1=2�Q1=2

h iW �I

2666664

3777775p0,

i ¼ 1; . . . ; L, (27)

which is equivalent to Eq. (25) with r2 ¼ 1.According to the above analysis, the robust model matchingcontrol of innate immune system via fuzzy interpolationmethod is summarized as follows.

Design procedure:

(1)

Give a desired reference model in Eq. (4) of immune system. (2) Select membership functions and construct fuzzy plant rules

in Eq. (17).

(3) Give weighting matrices Q and R in Eq. (5). (4) Solve the LMI-constrained optimization in Eq. (25) to obtain

W (thus P ¼W�1 can also be obtained) and r20.

(5)

Construct the controller under the worst-case disturbance inEq. (21).

Remark 3. The software packages such as Robust Control Toolboxin Matlab can be employed to solve the LMI-constrainedoptimization problem in Eq. (25) easily.

ARTICLE IN PRESS

B.-S. Chen et al. / Journal of Theoretical Biology 253 (2008) 824–837830

5. Computational simulation

Example 1. We consider the innate immune system in Eq. (1)and in Fig. 1. The values of the parameters are in Table 1. Theenvironmental disturbances w1–w4 are unknown but boundedsignals. Under infectious situation, the microbes infect the organnot only by an initial concentration at the beginning but also bythe continuous pathogens input. For the convenience of computersimulation, suppose the continuous pathogens input to theimmune system is viewed as an environmental disturbance w1

shown in Fig. 3. For the convenience of simulation, w2–w4 areassumed zero mean white noises with standard deviations allequal to 2. The stochastic noises of immune systems are mainlydue to measurement errors, modeling errors and process noises(Milutinovic and De Boer, 2007). The dynamic model of innateimmune system under exogenous pathogens input and environ-mental disturbances is controlled by a combined therapeuticcontrol shown as follows (Stengel et al., 2002a):

_x1 ¼ ð1� x3Þx1 � u1 þ 3w1,

_x2 ¼ a21ðx4Þ3x1x3 � ðx2 � 2Þ þ u2 þ 3w2,

Table 1Model parameters of dynamic innate immune system (Marchuk, 1983; Stengel

et al., 2002b)

Parameter Value Description

a11 1 Pathogens reproduction rate coefficient

a12 1 The suppression by pathogens coefficient

a22 3 Immune reactivity coefficient

a23 1 The mean immune cell production rate coefficient

x�2 2 The steady-state concentration of immune cells

a31 1 Antibodies production rate coefficient

a32 1.5 The antibody mortality coefficient

a33 0.5 The rate of antibodies suppress pathogens

a41 0.5 The organ damage depends on the pathogens damage

possibilities coefficient

a42 1 Organ recovery rate

b1 �1 Pathogen killer’s agent coefficient

b2 1 Immune cell enhancer coefficient

b3 1 Antibody enhancer coefficient

b4 �1 Organ health enhancer coefficient

Fig. 3. The continuous exogenous pathogens intrusion.

_x3 ¼ x2 � ð1:5þ x1Þx3 þ u3 þ 3w3,

_x4 ¼ 0:5x1 � x4 � u4 þ 3w4,

a21ðx4Þ ¼cosðpx4Þ; 0px4p1=2;

0; 1=2px4;

((28)

with the set of the initial condition x(0) ¼ ½ x1ð0Þ x2ð0Þ x3ð0Þx4ð0Þ�

T ¼ ½3 3:1 1 0:98 �T. In this example, therapeutic con-trols u1–u4 are combined to enhance the immune system.

Our reference model design objective is that system matrix Ar

and r(t) should be specified beforehand so that its transientresponses and steady state of reference system for innate immuneresponse system are desired. If the real parts of eigenvaluesof Ar are more negative (i.e. more robust stable), the trackingsystem will be more robust to environmental disturbances. Aftersome numerical simulations for clinical treatment, the desiredreference signals are obtained by the following reference model(see Fig. 4):

_xrðtÞ ¼

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

26664

37775� xrðtÞ þ Br � ustepðtÞ, (29)

where Br ¼ ½0 4:6 13:3333 0 �T and ustepðtÞ is the unit stepfunction.

From the investigation of the uncontrolled innate immuneresponse (lethal case) in Fig. 5, the pathogen concentration isincreasing rapidly and causes organ failure. We try to administratea treatment after a period of pathogens infection to enhancethe immune system. The cutting line (black solid line) inFig. 5 is a proper time to take drugs. Suppose the set of theinitial condition of the desired reference model is about xrð0Þ ¼½2:9 3:2 1:1 0:9 �T. The time response of the desired referencemodel in Eq. (29) is shown in Fig. 4.

To minimize the design effort and complexity for this non-linear innate immune system in Eq. (28), we employ the T–S fuzzymodel to construct fuzzy rules to approximate the nonlinearinnate immune system with the innate immune system’s statevariables as premise variables in the following.

Fig. 4. The desired reference model with four desired states in Eq. (29): pathogens

(xr1, blue, dashed square line), immune cells (xr2, green, dashed triangle line),

antibodies (xr3, red, dashed diamond line), and organ (xr4, magenta, dashed circle

line). The initial conditions of the reference model are xrð0Þ ¼ ½2:9 3:2 1:1 0:9 �T.

ARTICLE IN PRESS

B.-S. Chen et al. / Journal of Theoretical Biology 253 (2008) 824–837 831

�

P ¼

Rule i

If x1 is Fi1; x2 is Fi2; x3 is Fi3; and x4 is Fi4,

then _xðtÞ ¼ AixðtÞ þ BuðtÞ þ CvðtÞ; i ¼ 1;2;3; . . . ; L, (30)

where x ¼ ½ x1 x2 x3 x4 xr1 xr2 xr3 xr4 �T, u ¼ ½u1 u2 u3

u4�T, v ¼ ½w1 w2 w3 w4 r1 r2 r3 r4 �

T, the parameters Ai,B, C are described in Appendix C and the number of the fuzzy rulesis L ¼ 16. To construct the fuzzy model in Eq. (30), we need to findthe operating points of the innate immune response. Suppose theoperating points for x1 are at x11 ¼ 0 and x12 ¼ 4. Similarly, theoperating points of x2, x3, and x4 are at x21 ¼ 0, x22 ¼ 10, x31 ¼ 0,x32 ¼ 5, x41 ¼ 0, and x42 ¼ 1, respectively. For the convenience ofdesign, triangle-type membership functions are taken for Rule 1through Rule 16. We create two triangle type membershipfunctions for each state at these operating points (see Fig. 6). Inorder to accomplish the robust matching performance, we shouldtune up a set of the weighting matrices Q and R of the costfunction in Eq. (11) as follows:

Q ¼

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

26664

37775; R ¼

0:003 0 0 0

0 0:003 0 0

0 0 0:003 0

0 0 0 0:003

26664

37775.

After specifying the desired reference model, we need to solvethe constrained optimization in Eq. (24) for the robust minimaxcontrol in Eq. (21) by employing Matlab Robust Control Toolbox.Finally, we obtain a minimum attenuation level r2

0 ¼ 0:98 and acommon positive definite symmetric matrix P for Eq. (22) asfollows:

0:43313 0 0 0 �0:43313 0 0 0

0 0:56172 0 0 0 �0:56172 0 0

0 0 0:42678 0 0 0 �0:42678 0

0 0 0 0:28482 0 0 0 �0:28482

�0:43313 0 0 0 0:50151 0 0 0

0 �0:56172 0 0 0 0:62738 0 0

0 0 �0:42678 0 0 0 0:49526 0

0 0 0 �0:28482 0 0 0 0:34567

266666666666664

377777777777775

.

Figs. 7 and 8 present the simulation results for the robust modelmatching control. Fig. 7 shows the responses of the controlledimmune system by minimax model matching control with theconcentrations of the pathogens x1, immune cells x2, antibodies x3,and organ index x4 to track the desired reference states xr1, xr2, xr3,

P ¼

0:29932 0 0 0 �0:29932 0 0 0

0 0:39859 0 0 0 �0:39859 0 0

0 0 0:28408 0 0 0 �0:28408 0

0 0 0 0:30502 0 0 0 �0:30502

�0:29932 0 0 0 0:35217 0 0 0

0 �0:39859 0 0 0 0:43261 0 0

0 0 �0:28408 0 0 0 0:33404 0

0 0 0 �0:30502 0 0 0 0:33088

266666666666664

377777777777775

.

and xr4, respectively. From the simulation results, the trackingperformance of the robust model matching control via T–S fuzzyinterpolation is quite satisfactory. Fig. 8 shows the four combinedtherapeutic control signals. Obviously, from Figs. 8 and 9, it is seenthat the effect of stochastic external disturbances on the referencemodel tracking of immune system is attenuated significantly bythe proposed robust therapeutic control design.

Example 2. It is sometime unavoidable for drugs to haveadverse side effects. Limiting the impact of side effects should beconsidered in this therapeutic control design. Suppose thepathogen killer’s agent u1 (i.e. drugs or antibiotics) also degradesorgan health. The modified model of Eq. (28) of the organ state x4

becomes

_x4 ¼ 0:5x1 � x4 � u4 þ 0:2u1 þ 3w4, (31)

i.e. an increase of 0.2 side effect of the pathogen killer’s agent u1

on the organ state x4. The detailed design of fuzzy approximationis similar to the previous procedure and the parameters are thesame except B, i.e.

B ¼

�1 0 0 0

0 1 0 0

0 0 1 0

0:2 0 0 �1

0 0 0 0

0 0 0 0

0 0 0 0

0 0 0 0

266666666666664

377777777777775

.

In this design case, we choose

Q ¼

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 2

26664

37775; R ¼

0:002 0 0 0

0 0:002 0 0

0 0 0:002 0

0 0 0 0:002

26664

37775.

Since the organ health state suffers from the impact of the sideeffects, we should increase the weighting q44 to impose a penalty

on the organ health tracking error. Furthermore, we relax theweighting on the control signals to improve the trackingefficiency. By the same design procedure, we obtain a minimumattenuation level r2

0 ¼ 0:9 and a common positive definitesymmetric matrix P as follows:

The control design with considered drug’s side effects is simulatedin Figs. 9 and 10. Fig. 9 shows the responses of the controlledimmune system by the proposed minimax model matchingcontrol with undergoing the side effects of pathogen killer’s agentu1. From the simulation result, the organ health state x4 tracks thedesired response leisurely at the beginning but it totally matchesthe desired response at the end. Fig. 10 shows the four combined

ARTICLE IN PRESS

B.-S. Chen et al. / Journal of Theoretical Biology 253 (2008) 824–837832

therapeutic control signals with the side effects of pathogenkiller’s agent u1. At the beginning, the drug of organ healthenhancer u4 needs to increase to minimize the side effects fromthe pathogen killer’s agent u1.

6. Discussion

From the simulation results (Figs. 7 and 8), it is shown that theinnate immune system under the continuous intrusion ofexogenous pathogens and the corruption of environmentaldisturbances can be controlled by a robust model matchingcontrol design to achieve the desired time response. If we considerthe conventional optimal control in Eq. (3), i.e. the effect of theenvironmental disturbances is not included in the cost function;the optimal tracking control problem is equivalent to letting r2 ¼

1 in Eqs. (5) and (22) (Boyd, 1994). From the simulation results

Fig. 5. The uncontrolled immune responses (lethal case) in Eq. (28) are shown to

increase the level of pathogen concentration at the beginning of the time period. In

this case, we try to administrate a treatment after a short period of pathogens

infection. The cutting line (black solid line) is an optimal time point to give drugs.

The organ will survive or fail based on the organ health threshold (horizontal

dashed line) (x4o1: survival; x4X1: failure).

Fig. 6. Membership functions for

(Figs. 11 and 12), the four states of optimal tracking of the immunesystem are overshooting and diverging without tracking thedesired immune time response. Obviously, exogenous pathogensand the environmental disturbances have deteriorated theoptimal tracking performance and therefore their effects shouldbe considered in the robust control design procedure. In thesituation, the proposed robust matching control design isnecessary to achieve a desired time response.

Pathogen killer’s agent u1 has toxic side effect that can damagethe organ during the treatment. So, we should consider itsdetrimental impacts into the innate immune system in the modelin Eq. (31). In the robust model matching control designprocedure, the element of weighting matrix q44 should beincreased to impose a penalty on the tracking error of organhealth state to eliminate the side effect of u1 on x4. From thesimulation results (Figs. 9 and 10), the organ health state x4 tracksthe desired response slowly at the beginning but it almostmatches the desired response at the end. It is also observed thatthe innate immune system can be controlled by a combinedtherapies design to achieve the desired time response under the

four states x1, x2, x3, and x4.

Fig. 7. The tracking of innate immune system to the desired reference model by

the robust minimax matching control under the continuous exogenous pathogens

and environmental disturbances.

ARTICLE IN PRESS

Fig. 8. The minimax controls in the simulation example. The drug controls u1

(blue, solid square line) for pathogens, u2 for immune cells (green, solid triangle

line), u3 for antibodies (red, solid diamond line), and u4 for organ (magenta, solid

circle line).

Fig. 9. The tracking of innate immune system to the desired reference model by

the robust minimax matching control under the continuous exogenous pathogens,

environmental disturbances and pathogen killer’s agent side effects.

Fig. 10. The minimax controls with considering the pathogen killer’s agent side

effects. The drug controls u1 (blue, solid square line) for pathogens, u2 for immune

cells (green, solid triangle line), u3 for antibodies (red, solid diamond line), and u4

for organ (magenta, solid circle line).

Fig. 11. In the case of conventional optimal matching control (i.e. r ¼ 1 in Eq. (5)),

since the effect of continuous exogenous pathogens intrusion and environmental

disturbances is not considered in the design procedure, the states of innate

immune system overshoot and diverge and cannot track the desired reference

responses.

B.-S. Chen et al. / Journal of Theoretical Biology 253 (2008) 824–837 833

continuous intrusion of exogenous pathogens, environmentaldisturbances and side effects.

The combined therapies design is an important issue for allhuman diseases (Villadsen et al., 2003). For a long period, thetreatment of inflammatory skin diseases such as psoriasis, contactdermatitis and atopic dermatitis has included agents that alleviatesymptoms, but these agents have not been aimed at any specificmolecular targets involved in the pathogenesis of the disease.Insights into this immune mechanism may facilitate the devel-opment of combination therapies that take advantage of therobust model matching design, with the aim of achieving higherefficacy at a lower drug dosage and with a reduced probability ofside effect on organ health. The proposed robust model matchingdesign has used four control variables, i.e. pathogen killer’s agentu1, immune cell enhancer u2, antibody enhancer u3, and health

enhancer u4, to achieve a minimax matching performance and toefficiently attenuate the effect of exogenous pathogens andenvironmental disturbances on the immune system.

In this study, the model of innate immune dynamic system istaken from the literature, which still needs to compare quantita-tively with empirical evidence in practical application. Forpractical implementation, accurate biodynamic models are re-quired for treatment application. However, model identification isnot the topic of this paper. Furthermore, we have made anassumption that the four states (x1–x4) of the concentrations orindices can be measured accurately by the medical equipment.With these detectable signals, we can solve these dynamic gameproblems for robust tracking control design of innate immunesystem to obtain the drug administration values in real timethrough medical instrument readout. If measurement is corrupted

ARTICLE IN PRESS

Fig. 12. The controls of conventional optimal matching control (i.e. r ¼ 1 in

Eq. (5)) without considering the effect of continuous exogenous pathogens

intrusion and environmental disturbances in the design procedure, the drug

controls u1 (blue, solid square line) for pathogens, u2 for immune cells (green, solid

triangle line), u3 for antibodies (red, solid diamond line), and u4 for organ

(magenta, solid circle line) are all overshooting and divergent.

B.-S. Chen et al. / Journal of Theoretical Biology 253 (2008) 824–837834

by noises in the measurement process, some filter designs shouldbe employed to attenuate these noises to estimate the statevariables for control in Eq. (21) (Stengel and Ghigliazza, 2004).Nevertheless, the implementation of filter will increase thecomplexity of the design problem (Althaus et al., 2007). Sincethe proposed robust model matching control design can providean efficient way to create a real time therapeutic regime to protectsuspected patients from the pathogens infection, in the future, wewill focus on applications of robust control design to therapy anddrug design incorporating with nanotechnology and metabolicengineering scheme.

As a comparison, the similarity and difference between ourminimax control method and the optimal control method (Shudoand Iwasa, 2004) are given in the following. We all want tominimize the total cost of design, i.e. the weighted sum of thedamage caused by pathogens and the cost paid by specificimmune cells. On the other hand, the differences are given inthe following: (i) A desired model reference is given to beoptimally tracked for the enhancement of the immune system inour proposed method and Shudo and Iwasa have designed anoptimal control to minimize the cost. (ii) The effect of externaldisturbances has not been considered in Shudo and Iwasa, but theworst-case effect of external disturbances has been consideredand minimized in our design via the minimax Nash game method.(iii) Fuzzy interpolation technique is employed by our method sothat LMIs technique is used to efficiently solve the nonlinearminimax optimization problem in our design procedure. However,Shudo and Iwasa have used a dynamic programming method toderive an optimal schedule to solve the nonlinear optimizationproblem of host defense, immune memory and post-infectionpathogen levels in mammals.

7. Conclusion

Robustness is a significant property that allows the innateimmune system to maintain its function despite exogenouspathogens, environmental disturbances and system uncertainties.Based on dynamic game theory, the robust tracking control is

formulated as a minimax problem for an innate immune system toachieve a desired time response prescribed prior under environ-mental disturbances, unknown initial conditions and side effects. Ingeneral, the robust model matching control design for innateimmune system needs to solve nonlinear HJI, which is generallydifficult to solve for this control design. Based on the proposedfuzzy dynamic game scheme, the design of nonlinear dynamicrobust matching control problem for innate immune system istransformed to solve a set of equivalent linear dynamic gameproblem. Such transformation can then allow us an easier approachby solving an LMI-constrained optimization problem for robustminimax control design. With the help of the Robust ControlToolbox in Matlab instead of the HJI, we could solve these lineardynamic game problems for robust matching control of innateimmune system efficiently. From the in silico simulation examples,the proposed minimax match control of immune system couldtrack the prescribed reference time response robustly, which maylead to potential application in therapeutic drug design for adesired immune response during an infection episode.

Acknowledgment

The study is supported by the National Science Council ofTaiwan under Grant no. NSC 96-2627-B-007-004.

Appendix A. Proof of Theorem 1

Let us denote a Lyapunov energy function VðxðtÞÞ40. Then Eq.(9) is equivalent to the following minimax problem:

minuðtÞ

maxvðtÞ

J ¼ minuðtÞ

maxvðtÞ

Vðxð0ÞÞ � Vðxðtf ÞÞ�

þ

Z tf

0xTðtÞQ xðtÞ þ uTðtÞRuðtÞ

�r2vTðtÞvðtÞ þ

dVðxðtÞÞ

dt

�dt

�; 8xð0Þ. (A.1)

By the chain rule, we get

dVðxðtÞÞ

dt¼

qVðxðtÞÞ

qxðtÞ

� �T dxðtÞ

dt

¼qVðxðtÞÞ

qxðtÞ

� �T

ðFðxðtÞÞ þ BuðtÞ þ CvðtÞÞ. (A.2)

Substituting Eq. (A.2) into Eq. (A.1), we get

minuðtÞ

maxvðtÞ

J ¼ minuðtÞ

maxvðtÞ

Vðxð0ÞÞ � Vðxðtf ÞÞ�

þ

Z tf

0xTðtÞQ xðtÞ þ uTðtÞRuðtÞ � r2vTðtÞvðtÞ

þqVðxðtÞÞ

qxðtÞ

� �T

FðxðtÞÞ þqVðxðtÞÞ

qxðtÞ

� �T

�BuðtÞ þqVðxðtÞÞ

qxðtÞ

� �T

CvðtÞ

#dt

); 8xð0Þ.

minuðtÞ

maxvðtÞ

J ¼ minuðtÞ

maxvðtÞ

Vðxð0ÞÞ � Vðxðtf ÞÞ�

þ

Z tf

0

qVðxðtÞÞ

qxðtÞ

� �T

FðxðtÞÞ þ xðtÞTQ xðtÞ

"

�1

4

qVðxðtÞÞ

qxðtÞ

� �T

BR�1BT qVðxðtÞÞ

qxðtÞ

þ1

4r2

qVðxðtÞÞ

qxðtÞ

� �T

CCT qVðxðtÞÞ

qxðtÞ

þ uTðtÞ þ1

2R�1BT qVðxðtÞÞ

qxðtÞ

� �T

ARTICLE IN PRESS

B.-S. Chen et al. / Journal of Theoretical Biology 253 (2008) 824–837 835

� R uðtÞ þ1

2R�1BT qVðxðtÞÞ

qxðtÞ

� �

� rvðtÞ �1

2rCT qVðxðtÞÞ

qxðtÞ

� �T

� rvðtÞ �1

2rCT qVðxðtÞÞ

qxðtÞ

� ��dt

�; 8xð0Þ.

Therefore, the minimax solution is given as follows:

Jðu�ðtÞ; v�ðtÞÞ ¼ Vðxð0ÞÞ � Vðxðtf ÞÞ

(

þ

Z tf

0

qVðxðtÞÞ

qxðtÞ

� �T

FðxðtÞÞ þ xðtÞTQ xðtÞ

"

�1

4

qVðxðtÞÞ

qxðtÞ

� �T

BR�1BT qVðxðtÞÞ

qxðtÞ

þ1

4r2

qVðxðtÞÞ

qxðtÞ

� �T

CCT qVðxðtÞÞ

qxðtÞ

#dt

); 8xð0Þ

with

u�ðtÞ ¼ �1

2R�1BT qVðxðtÞÞ

qxðtÞ; v�ðtÞ ¼

1

2r2CT qVðxðtÞÞ

qxðtÞ.

If Eq. (14) holds, then

Jðu�ðtÞ; v�ðtÞÞpVðxð0ÞÞ � Vðxðtf ÞÞ.

From the inequality in Eq. (11), this minimax solution should beless than r2xT

ð0ÞIxð0Þ, and then we get the inequality in Eq. (15):

minuðtÞ

maxvðtÞ

JpVðxð0ÞÞ � Vðxðtf ÞÞpVðxð0ÞÞ

pr2xTð0ÞIxð0Þ; 8xð0Þ.

Appendix B. Proof of Theorem 2

Let us denote a Lyapunov energy function VðxðtÞÞ ¼

xTðtÞPxðtÞ40. Then Eq. (9) is equivalent to the following:

minuðtÞ

maxvðtÞ

J ¼ minuðtÞ

maxvðtÞ

xTð0ÞPxð0Þ � xTðtf ÞPxðtf Þ

�

þ

Z tf

0xTðtÞQ xðtÞ þ uTðtÞRuðtÞ

�

�r2vT ðtÞvðtÞ þdVðxðtÞÞ

dt

�dt

�; 8xð0Þ

¼ minuðtÞ

maxvðtÞ

xTð0ÞPxð0Þ � xTðtf ÞPxðtf Þ

�

þ

Z tf

0ðxTðtÞQ xðtÞ þ uTðtÞRuðtÞ

�r2vTðtÞvðtÞ þ 2xTðtÞP _xðtÞdtÞ

; 8xð0Þ

¼ minuðtÞ

maxvðtÞ

xTð0ÞPxð0Þ � xT

ðtf ÞPxðtf Þ�

þ

Z tf

0xTðtÞQ xðtÞ þ uTðtÞRuðtÞ � r2vTðtÞvðtÞ

þ 2xTðtÞPXL

i¼1

hiðxðtÞÞAixðtÞ

!

þ2xTðtÞPBuðtÞ þ 2xT

ðtÞPCvðtÞ�dt; 8xð0Þ

¼ minuðtÞ

maxvðtÞ

xTð0ÞPxð0Þ � xT

ðtf ÞPxðtf Þ�

þ

Z tf

0xTðtÞQ xðtÞ þ uTðtÞRuðtÞ � r2vTðtÞvðtÞ

þXL

i¼1

hiðxðtÞÞð2xTðtÞPAixðtÞ þ 2xT

ðtÞPBuðtÞ

þ2xTðtÞPCvðtÞÞ

�dt; 8xð0Þ

¼ minuðtÞ

maxvðtÞ

xTð0ÞPxð0Þ � xT

ðtf ÞPxðtf Þ�

þ

Z tf

0xTðtÞQ xðtÞ þ

XL

i¼1

hiðxðtÞÞxTðtÞ

"

� PAi þ ATi P � PTBR�1BTP þ

1

r2PTCCTP

� �xðtÞ

þ ½uðtÞ þ R�1BT PxðtÞ�TR½uðtÞ þ R�1BTPxðtÞ�

� rvðtÞ �1

rCTPxðtÞ

� �T

� rvðtÞ �1

rCT PxðtÞ

� ��dt

�; 8xð0Þ.

The minimax solution is given as follows:

Jðu�ðtÞ; v�ðtÞÞ ¼ xTð0ÞPxð0Þ � xTðtf ÞPxðtf Þ

�

þ

Z tf

0xTðtÞQ xðtÞ þ

XL

i¼1

hiðxðtÞÞxTðtÞ

"

� PAi þ ATi P � PTBR�1BTP

�þ

1

r2PTCCTP

�xðtÞ

�dt

�; 8xð0Þ

with

u�ðtÞ ¼ �R�1BTPxðtÞ; v�ðtÞ ¼1

r2CTPxðtÞ.

In order to simplify the above equation, suppose the inequality inEq. (22) holds, then

minuðtÞ

maxvðtÞ

Jpxð0ÞPxð0Þ; 8xð0Þ.

From the inequality in Eq. (11), this minimax should be less thanr2xTð0ÞIxð0Þ, and then

minuðtÞ

maxvðtÞ

Jpxð0ÞPxð0Þpr2xTð0ÞIxð0Þ; 8xð0Þ

i:e:; Ppr2I.

Since we assume r2 is the upper bound in Eq. (5), the minimaxcontrol becomes how to design u�ðtÞ in Eq. (21) by solving theconstrained optimization problem in Eqs. (22) and (23).

Appendix C. Parameters of the fuzzy system

The nonlinear innate immune system in Eq. (28) could beapproximated by a T–S fuzzy system. By the fuzzy modelingmethod (Takagi and Sugeno, 1985), the matrices of the local linearsystem Ai, the parameters B, and C are calculated as follows:

A1 ¼

0:3462 0 0 0

3:3258 �1:1155 0:3433 2:1987

�0:2674 1:1816 �1:7724 0:1108

0:6295 0 0 �1:3994

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A2 ¼

0:348 0 0 0

�2:8398 �0:9076 �1:4305 �1:9826

�0:235 1:1644 �1:6635 0:1108

0:295 �0:0902 �0:1643 �1:3994

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

ARTICLE IN PRESS

B.-S. Chen et al. / Journal of Theoretical Biology 253 (2008) 824–837836

A3 ¼

�2:3807 0 0 0

10:616 �0:9107 0:3194 �1:6078

�2:4789 0:5007 �1:7724 �0:4134

0:4117 0 0 �0:5537

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A4 ¼

�2:2996 0 0 0

�8:4642 �0:9535 �0:4587 1:1711

�2:3919 0:4936 �1:6635 �0:4134

0:1957 �0:0781 �0:1643 �0:5537

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A5 ¼

0:1987 0 0 0

0:6956 �0:933 0:3733 �3:5255

0:3935 1:1816 �0:4106 0:1732

0:2311 0 0 �0:3611

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A6 ¼

0:1981 0 0 0

�2:9804 �1:0591 �2:0136 3:3337

0:3963 1:1644 �0:3837 0:1732

0:1101 �0:0902 �0:0619 �0:3611

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A7 ¼

�1:0821 0 0 0

5:3534 �0:7742 �0:6384 �0:7012

�0:7249 0:5007 �0:4106 �0:0417

0:1682 0 0 �0:1482

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A8 ¼

�1:057 0 0 0

�2:8391 �1:532 0:7299 0:2814

�0:7023 0:4936 �0:3837 �0:0417

0:0812 �0:0781 �0:0619 �0:1482

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A9 ¼

0:3462 0:2243 �2:2977 0:087

3:0103 �0:0287 7:6732 �2:1137

�0:2674 0:9206 �2:1832 �0:0587

0:6295 0:1473 0:1675 0:0737

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A10 ¼

0:348 0:2223 �2:224 0:087

�2:4956 �1:8301 �5:6794 0:7798

�0:235 0:9101 �2:1142 �0:0587

0:295 0:0728 0:0811 0:0737

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A11 ¼

�2:3807 �0:9739 �2:2977 �0:5749

12:767 2:4941 11:649 �1:196

�2:4789 �0:2104 �2:1832 �0:598

0:4117 0:1305 0:1675 0:0489

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A12 ¼

�2:2996 �0:9641 �2:224 �0:5749

�11:901 �3:8172 �11:21 �3:8954

�2:3919 �0:2087 �2:1142 �0:598

0:1957 0:0646 0:0811 0:0489

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A13 ¼

0:1987 0:2243 �1:0582 0:0495

0:5515 �0:6453 3:8238 2:1633

0:3935 0:9206 �0:7957 0:0991

0:2311 0:1473 0:0772 0:0275

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A14 ¼

0:1981 0:2223 �1:0349 0:0495

0:1057 �1:2516 �3:148 �2:882

0:3963 0:9101 �0:7759 0:0991

0:1101 0:0728 0:0377 0:0275

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A15 ¼

�1:0821 �0:9739 �1:0582 �0:2642

3:6655 3:8523 4:0278 4:4254

�0:7249 �0:2104 �0:7957 �0:1756

0:1682 0:1305 0:0772 0:0203

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

A16 ¼

�1:057 �0:9641 �1:0349 �0:2642

�6:4434 �4:5651 �4:4027 �6:7461

�0:7023 �0:2087 �0:7759 �0:1756

0:0812 0:0646 0:0377 0:0203

0

0

�12 0 0 0

0 �2:3 0 0

0 0 �10 0

0 0 0 �3

2666666666666664

3777777777777775

,

ARTICLE IN PRESS

B.-S. Chen et al. / Journal of Theoretical Biology 253 (2008) 824–837 837

B ¼

�1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 �1

0 0 0 0

0 0 0 0

0 0 0 0

0 0 0 0

266666666666664

377777777777775; C ¼

3 0 0 0

0 3 0 0

0 0 3 0

0 0 0 3

0

0

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

2666666666666664

3777777777777775

.

References

Adler, F.R., Karban, R., 1994. Defended fortresses or moving targets? Another modelof inducible defenses inspired by military metaphors. Am. Nat. 144, 813–832.

Althaus, C.L., Ganusov, V.V., De Boer, R.J., 2007. Dynamics of CD8+ T cell responsesduring acute and chronic lymphocytic choriomeningitis virus infection.J. Immunol. 179, 2944–2951.

Asachenkov, A.L., 1994. Disease Dynamics. Birkhauser, Boston.Astrom, K.J., Wittenmark, B., 1995. Adaptive Control. Addison-Wesley, Reading, MA.Basar, T., Olsder, G.J., 1999. Dynamic Noncooperative Game Theory. SIAM,

Philadelphia, PA.Bell, D.J., Katusiime, F., 1980. A time-optimal drug displacement problem. Optim.

Control Appl. Meth. 1, 217–225.Bellman, R., 1983. Mathematical Methods in Medicine. World Scientific, Singapore.Bonhoeffer, S., May, R.M., Shaw, G.M., Nowak, M.A., 1997. Virus dynamics and drug

therapy. Proc. Natl. Acad. Sci. USA 94, 6971–6976.Boyd, S.P., 1994. Linear Matrix Inequalities in System and Control Theory. Society

for Industrial and Applied Mathematics, Philadelphia, PA.Carson, E.R., Cramp, D.G., Finkelstein, F., Ingram, D., 1985. Control system concepts

and approaches in clinical medicine. In: Carson, E.R., Cramp, D.G. (Eds.),Computers and Control in Clinical Medicine. Plenum Press, New York, pp. 1–26.

Chen, B.S., Tseng, C.S., Uang, H.J., 1999. Robustness design of nonlinear dynamicsystems via fuzzy linear control. IEEE Trans. Fuzzy Syst. 7, 571–585.

Chen, B.S., Tseng, C.S., Uang, H.J., 2000. Mixed H-2/H-infinity fuzzy output feedbackcontrol design for nonlinear dynamic systems: an LMI approach. IEEE Trans.Fuzzy Syst. 8, 249–265.

Chizeck, H.J., Katona, P.G., 1985. Closed-loop control. In: Carson, E.R., Cramp, D.G. (Eds.),Computers and Control in Clinical Medicine. Plenum Press, New York, pp. 95–151.

De Boer, R.J., Boucher, C.A., 1996. Anti-CD4 therapy for AIDS suggested bymathematical models. Proc. Biol. Sci. 263, 899–905.

Gentilini, A., Morari, M., Bieniok, C., Wymann, R., Schnider, T.W., 2001. Closed-loopcontrol of analgesia in humans. In: Proceedings of IEEE Conference on Decisionand Control, vol. 1, Orlando, pp. 861–866.

Janeway, C., 2005. Immunobiology: The Immune System in Health and Disease.Garland, New York.

Jelliffe, R.W., 1986. Clinical applications of pharmacokinetics and control theory:planning, monitoring, and adjusting regimens of aminoglycosides, lidocaine,digitoxin, and digoxin. In: Maronde, R.F. (Ed.), Topics in Clinical Pharmacologyand Therapeutics. Springer, New York, pp. 26–82.

Kirschner, D., Lenhart, S., Serbin, S., 1997. Optimal control of the chemotherapy ofHIV. J. Math. Biol. 35, 775–792.

Kwong, G.K., Kwok, K.E., Finegan, B.A., Shah, S.L., 1995. Clinical evaluation of longrange adaptive control for mean arterial blood pressure regulation. In:Proceedings of American Control Conference, vol. 1, Seattle, pp. 786–790.

Li, T.H.S., Chang, S.J., Tong, W., 2004. Fuzzy target tracking control of autonomousmobile robots by using infrared sensors. IEEE Trans. Fuzzy Syst. 12, 491–501.

Lian, K.Y., Chiu, C.S., Chiang, T.S., Liu, P., 2001. LMI-based fuzzy chaoticsynchronization and communications. IEEE Trans. Fuzzy Syst. 9, 539–553.

Lydyard, P.M., Whelan, A., Fanger, M.W., 2000. Instant Notes in Immunology.Springer, New York.

Marchuk, G.I., 1983. Mathematical Models in Immunology. Optimization Software,Inc. Worldwide distribution rights by Springer, New York.

Milutinovic, D., De Boer, R.J., 2007. Process noise: an explanation for thefluctuations in the immune response during acute viral infection. Biophys.J. 92, 3358–3367.

Nowak, M.A., May, R.M., 2000. Virus Dynamics: Mathematical Principles ofImmunology and Virology. Oxford University Press, Oxford.

Nowak, M.A., May, R.M., Phillips, R.E., Rowland-Jones, S., Lalloo, D.G., McAdam, S.,Klenerman, P., Koppe, B., Sigmund, K., Bangham, C.R., et al., 1995. Antigenicoscillations and shifting immunodominance in HIV-1 infections. Nature 375,606–611.

Parker, R.S., Doyle III, J.F., Harting, J.E., Peppas, N.A., 1996. Model predictive controlfor infusion pump insulin delivery. In: Proceedings of the 18th AnnualInternational Conference of the IEEE Engineering in Medicine and BiologySociety, vol. 5, Amsterdam, pp. 1822–1823.

Perelson, A.S., Weisbuch, G., 1997. Immunology for physicists. Rev. Modern Phys.69, 1219–1267.

Perelson, A.S., Kirschner, D.E., De Boer, R., 1993. Dynamics of HIV infection of CD4+T cells. Math. Biosci. 114, 81–125.

Perelson, A.S., Neumann, A.U., Markowitz, M., Leonard, J.M., Ho, D.D., 1996. HIV-1dynamics in vivo: virion clearance rate, infected cell life-span, and viralgeneration time. Science 271, 1582–1586.

Polycarpou, M.M., Conway, J.Y., 1995. Modeling and control of drug deliverysystems using adaptive neuralcontrol methods. In: Proceedings of AmericanControl Conference, vol. 1, Seattle, pp. 781–785.

Robinson, D.C., 1986. Topics in clinical pharmacology and therapeutics. In:Maronde, R.F. (Ed.), Principles of Pharmacokinetics. Springer, New York, pp.1–12.

Rundell, A., HogenEsch, H., DeCarlo, R., 1995. Enhanced modeling of the immunesystem to incorporate natural killer cells and memory. In: Proceedings ofAmerican Control Conference, vol. 1, Seattle, pp. 255–259.

Schumitzky, A., 1986. Stochastic control of pharmacokinetic systems. In: Maronde,R.F. (Ed.), Topics in Clinical Pharmacology and Therapeutics. Springer,New York, pp. 13–25.

Shudo, E., Iwasa, Y., 2001. Inducible defense against pathogens and parasites:optimal choice among multiple options. J. Theor. Biol. 209, 233–247.

Shudo, E., Iwasa, Y., 2002. Optimal defense strategy: storage vs. new production.J. Theor. Biol. 219, 309–323.

Shudo, E., Iwasa, Y., 2004. Dynamic optimization of host defense, immunememory, and post-infection pathogen levels in mammals. J. Theor. Biol. 228,17–29.

Shudo, E., Haccou, P., Iwasa, Y., 2003. Optimal choice between feedforward andfeedback control in gene expression to cope with unpredictable danger.J. Theor. Biol. 223, 149–160.

Stafford, M.A., Corey, L., Cao, Y., Daar, E.S., Ho, D.D., Perelson, A.S., 2000. Modelingplasma virus concentration during primary HIV infection. J. Theor. Biol. 203,285–301.

Stengel, R.F., Ghigliazza, R., 2004. Stochastic optimal therapy for enhanced immuneresponse. Math. Biosci. 191, 123–142.

Stengel, R.F., Ghigliazza, R., Kulkarni, N., Laplace, O., 2002a. Optimal control ofinnate immune response. Optim. Control Appl. Meth. 23, 91–104.

Stengel, R.F., Ghigliazza, R.M., Kulkarni, N.V., 2002b. Optimal enhancement ofimmune response. Bioinformatics 18, 1227–1235.

Swan, G.W., 1981. Optimal-control applications in biomedical-engineering—asurvey. Optim. Control Appl. Meth. 2, 311–334.

Takagi, T., Sugeno, M., 1985. Fuzzy identification of systems and its applications tomodeling and control. IEEE Trans. Syst. Man Cybernet. 15, 116–132.

van Rossum, J.M., Steyger, O., van Uem, T., Binkhorst, G.J., Maes, R.A.A., 1986.Pharmacokinetics by using mathematical systems dynamics. In: Eisenfeld, J.,Witten, M. (Eds.), Modelling of Biomedical Systems. Elsevier, Amsterdam,pp. 121–126.

Villadsen, L.S., Skov, L., Baadsgaard, O., 2003. Biological response modifiers andtheir potential use in the treatment of inflammatory skin diseases. Exp.Dermatol. 12, 1–10.

Wein, L.M., D’Amato, R.M., Perelson, A.S., 1998. Mathematical analysis ofantiretroviral therapy aimed at HIV-1 eradication or maintenance of low viralloads. J. Theor. Biol. 192, 81–98.

Wiener, N., 1948. Cybernetics; or, Control and Communication in the Animal andthe Machine. Technology Press, Cambridge.

Wodarz, D., Nowak, M.A., 1999. Specific therapy regimes could lead tolong-term immunological control of HIV. Proc. Natl. Acad. Sci. USA 96,14464–14469.

Wodarz, D., Nowak, M.A., 2000. CD8 memory, immunodominance, and antigenicescape. Eur. J. Immunol. 30, 2704–2712.

Zhou, K., Doyle, J.C., Glover, K., 1996. Robust and Optimal Control. Prentice-Hall,Upper Saddle River, NJ.