RESEARCH Open Access

Adaptive reference update (ARU) algorithmA stochastic search algorithm for efficientoptimization of multi-drug cocktailsMansuck Kim Byung-Jun Yoon

From IEEE International Workshop on Genomic Signal Processing and Statistics (GENSIPS) 2011San Antonio TX USA 4-6 December 2011

Abstract

Background Multi-target therapeutics has been shown to be effective for treating complex diseases andcurrently it is a common practice to combine multiple drugs to treat such diseases to optimize the therapeuticoutcomes However considering the huge number of possible ways to mix multiple drugs at differentconcentrations it is practically difficult to identify the optimal drug combination through exhaustive testing

Results In this paper we propose a novel stochastic search algorithm called the adaptive reference update (ARU)algorithm that can provide an efficient and systematic way for optimizing multi-drug cocktails The ARU algorithmiteratively updates the drug combination to improve its response where the update is made by comparing theresponse of the current combination with that of a reference combination based on which the beneficial updatedirection is predicted The reference combination is continuously updated based on the drug response valuesobserved in the past thereby adapting to the underlying drug response function To demonstrate the effectivenessof the proposed algorithm we evaluated its performance based on various multi-dimensional drug functions andcompared it with existing algorithms

Conclusions Simulation results show that the ARU algorithm significantly outperforms existing stochastic searchalgorithms including the Gur Game algorithm In fact the ARU algorithm can more effectively identify potent drugcombinations and it typically spends fewer iterations for finding effective combinations Furthermore the ARUalgorithm is robust to random fluctuations and noise in the measured drug response which makes the algorithmwell-suited for practical drug optimization applications

BackgroundBiological networks are known to be redundant at variouslevels which makes them robust to various types of per-turbations As a consequence it is generally difficult tochange their long-term dynamics by blocking a specificgene or intervening in a specific pathway This is one ofthe reasons why monotherapy is often not very effective intreating complex diseases such as cancer and diabetes Infact multi-target therapeutics based on combinatory drugsare known to be much more effective and they are com-monly used these days for treating various diseases [1-6]

However considering the huge number of possible waysto mix multiple drugs it is practically impossible to findthe optimal ldquodrug cocktailrdquo simply by trial and error or byexhaustive testing Clearly we need a systematic way ofidentifying the most effective drug cocktail and recentlyseveral algorithms have been proposed to address the pro-blem of combinatorial drug optimization [7-12]For example Calzolari et al [7] developed a drug optimi-

zation algorithm based on sequential decoding algorithmsthat have been traditionally used in digital communica-tions [1314] In [7] it was shown that we can algorithmi-cally identify the optimal drug combination by testing onlya relatively small number of drug combinations comparedto exhaustive search Unlike the approach proposed by

Correspondence bjyoonecetamueduDepartment of Electrical and Computer Engineering Texas AampM UniversityCollege Station TX 77843-3128 USA

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

copy 2012 Kim and Yoon licensee BioMed Central Ltd This is an open access article distributed under the terms of the CreativeCommons Attribution License (httpcreativecommonsorglicensesby20) which permits unrestricted use distribution andreproduction in any medium provided the original work is properly cited

Calzolari et al [7] which was deterministic Wong et al [9]proposed a different approach based on a stochastic searchalgorithm called the Gur Game algorithm [1516] In thiswork [9] they formed a closed-loop optimization frame-work in which the Gur Game algorithm was used to pre-dict an updated drug combination that is likely to improvethe current drug response and the drug combination isiteratively updated until the response is maximized It wasshown that this closed-loop optimization method canquickly identify potent drug combinations More recentlyanother stochastic search algorithm was proposed in [11]that addresses the limitations of the Gur Game algorithmthereby further improving the performance of the closed-loop optimization approach originally proposed in [9]In this paper we propose a novel stochastic search algo-

rithm called the adaptive reference update (ARU) algo-rithm that can significantly improve the performance ofthe existing stochastic search algorithms [911] The keyidea of this algorithm is to adaptively update the referencedrug combination to guide the search algorithm and helpit to make better informed guesses without requiringextensive prior knowledge of the underlying biological net-work We demonstrate that the proposed ARU algorithmoutperforms existing stochastic drug optimization algo-rithms in terms of both efficiency success rate androbustness

MethodsCombinatorial drug optimization problemSuppose we have N different types of drugs where eachdrug can take one of pre-specified concentrations Ourgoal is to predict the optimal drug cocktail by mixing theavailable drugs that maximizes the overall therapeuticeffect Let xn be the concentration of the n-th drugwhere xn can take one of Mn distinct concentrations in

the set Cn = c1n c2n c3n cMnn where ckn lt ck+1n for all

k The drug cocktail is represented by an N-dimensionalvector x = (x1 x2 xN) which consists of the N drugconcentrations Let f(x) be the normalized drug responsethat quantitatively measures the effectiveness of a givendrug combination x We assume the response has beennormalized such that 0 le f (x) le 1 for x isin X whereX = C1 times C2 times middot middot middot times CN is the set of all possible drugcombinations We denote the number of all possible

drug combinations as M = |X | =prodN

n=1Mn f (x) = 0

implies that the drug cocktail x is completely ineffectiveand larger f (x) implies higher efficacy In practical appli-cations f (x) may be obtained by monitoring the cellresponse to a drug intervention using fluorescent ima-ging microarrays or sequencing techniques Under thissetting we aim to find the optimal drug combination xthat maximizes the normalized drug response

xlowast = argmax f (x)xisinX

As we can see this is a combinatorial optimizationproblem in which we have to find the optimal drugcombination out of M1 M2 MN possible combina-tions The total number of distinct drug combinationsquickly grows as the number of drugs increases Consid-ering the practical cost of experimentally measuring thenormalized drug response function f (x) it is apparentthat we cannot test all drug combinations to find themost effective one

Stochastic search algorithmsStochastic search algorithms [911] aim to efficientlyidentify the potent drug combinations without exploringthe entire combinatorial solution space The basic idea isto randomly search through the solution space by itera-tively updating the drug combination until an effectivecombination emerges At each step the current drugcombination is incrementally updated towards the direc-tion that is likely to improve the overall drug responseThe updating decision is made in a stochastic mannerwhich allows the search to proceed towards directionsthat are deemed to be less likely to improve the responseThis is an important characteristic of stochastic searchalgorithms which is critical for keeping the search frombeing trapped in local maxima Since a stochastic searchalgorithm tries to arrive at the optimal solution (ie themost effective drug combination) by performing iterativelocal searches its overall performance depends on how itchooses the next solution state (ie an updated drugcombination in X the set of all possible combinations)from a given state (ie the current drug combination)The two performance metrics of interest are (i) the effec-tiveness of the predicted drug combination in terms ofhow close its response is to the optimal response and (ii)the number of search steps that the algorithm needs totake until an effective combination is found Basically wewant to predict a potent drug cocktail by testing minimalnumber of drug combinations to minimize the actualexperimental cost for measuring the cell response tocombinatorial drugs When choosing the next state thesearch algorithm has to be as parsimonious as possiblein terms of the number of function evaluations so thatthe overall experimental cost for identifying the optimaldrug combination can be minimized This has been oneof the main design considerations of existing stochasticsearch algorithms that have been developed for combina-torial drug optimization [911]For example the Gur Game algorithm adopted in

[9] determines how to update the drug combinationsolely based on the current drug response Supposexc = (xc1 x

c2 middot middot middot xcN) is the current drug combination

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 2 of 15

with a normalized drug response of f (xc) The algo-rithm generates N random numbers rn Icirc [01] onefor each drug Each rn is compared against the currentdrug response xc = (xc1 x

c2 middot middot middot xcN) which is used to

either ldquorewardrdquo or ldquopenalizerdquo the n-th drug For exam-ple the drug is rewarded if f (xc) gtrn Otherwise it ispenalized This update process is repeated for each ofthe N drugs According to this scheme drug combina-tions with higher response are more likely to berewarded while combinations with lower response aremore likely to be penalized In the long run the algo-rithm is expected to drive the concentration of eachdrug towards the one that maximizes the responseNow an important relevant question is what is theright way of rewarding (or penalizing) the currentconcentration of a given drug The Gur Game algo-rithm used in [9] uses a predetermined finite stateautomaton (FSA) for this purpose For example letxc = ckn isin Cn be the current concentration of the n-thdrug Suppose we have f (xc) gtrn hence the currentconcentration xcn of the n-th drug should be rewardedThe FSA in [9] is designed such that the drug concen-tration is increased further if the current concentra-tion xcn is larger than a reference concentration crefn and decreased further if xcn is smaller than crefn Morespecifically if xcn = ckn and f(xc) gtrn then the drug con-centration is updated as follows

xcn =

⎧⎪⎪⎨⎪⎪⎩

ck+1n if xcn gt crefn and k lt Mn

ckn if xcn gt crefn and k = Mn

ckminus1n if xcn lt crefn and k gt 1ckn if xcn lt crefn and k = 1

(1)

The reference concentration crefn is typically chosen asthe median of the set Cn As shown in (1) the drugconcentration remains unchanged if it cannot beincreased (or decreased) further Now suppose that f

(xc) ltrn and therefore the current concentration xcn = cknshould be penalized In this case the concentration isupdated in the opposite direction

xcn =ckminus1n if xcn gt crefnck+1n if xcn lt crefn

(2)

Note that penalization moves the current drug con-

centration closer to the reference concentration crefn As

previously discussed in [11] one of the weaknesses ofthe Gur Game algorithm is that it uses a predeterminedFSA for updating (ie rewardingpenalizing) the drugconcentrations and does not adapt to the drug responsefunction at hand which is not known in advance As aresult the algorithm may perform poorly unless thedrug response function f(x) is properly normalized and

the reference concentration crefn is chosen adequately for

each drug For example consider the one-dimensionaldrug response f(x) shown in Figure 1A As we can seethe drug response has been over-normalized hence f(x) lt05 for any allowed concentration x Icirc [cmin cmax] Sincef(x) lt 05 a uniformly distributed random number r Icirc[01] is more likely to be larger than f(x) This impliesthat the Gur Game algorithm always tends to penalizethe current drug concentration (no matter what its valueis) which will probabilistically drive the concentrationtowards cref although it is clearly not optimal Figure 1Bshows another drug response for which the Gur Gamealgorithm will not work properly In this example wehave f(x) gt 05 hence the Gur Game algorithm is alwaysmore likely to reward the current drug concentrationwhich tends to drive the concentration away from thereference concentration cref This will push the concen-tration either towards cmin or cmax both of which aresuboptimalThe enhanced stochastic algorithm proposed in [11]

addresses this problem by making the search algorithmadapt to a given drug response The basic idea of thisalgorithm is to make an ldquoinformed-guessrdquo about how tobeneficially update a given drug concentration insteadof following a predetermined update rule Unlike theGur Game algorithm in [9] where all N drugs aresimultaneously updated based on the (same) currentdrug response f(xc) the enhanced algorithm updates theconcentration of one drug at a time As an examplesuppose during the last update of the n-th drug thedrug combination has been updated as

x = (x1 middot middot middot middot middot middot xN) rArr xprime = (xprime1 middot middot middot middot middot middot xprime

N)

where x and xprime are identical except for the concentra-

tion of the n-th drug hence xi = xprimei(i = n)and

xi = xprimei(i = n) We assume that xn and xprime

n differ only by

a single concentration level so that xn = ckn and

xprimen = ck+1n or xn = ck+1n and xprime

n = ckn for some k The algo-

rithm compares the two drug responses f (x) and f (xrsquo) thereby determine wether it would be more beneficial tofurther increase or decrease the concentration of then-th drug For example we may have the following fourcases

(Case - 1) xn lt xprimen and f (x) lt f (xprime) increasing the concentration is more beneficial

(Case - 2) xn gt xprimen and f (x) gt f (xprime) increasing the concentration is more beneficial

(Case - 3) xn lt xprimen and f (x) gt f (xprime) decreasing the concentration is more beneficial

(Case - 4) xn gt xprimen and f (x) lt f (xprime) decreasing the concentration is more beneficial

(3)

The above rules allow the algorithm to adaptivelydetermine how to reward (or penalize) a given drug con-centration based on the observed drug response valuesHowever the decision whether to reward or penalize the

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 3 of 15

current drug is made in a probabilistic manner For thispurpose we evaluate the following function

g(x xprime) =12

1 + α middot max

[f (x) f (xprime)

] (4)

where a Icirc [0 1] is a control parameter that adjuststhe randomness of the algorithm [11] This g(x xrsquo) iscompared with a uniformly distributed random numberrn Icirc [0 1] If g(x xrsquo) gt rn the n-th drug is rewardedie updated in such a way that appears to be more ben-eficial for enhancing the drug response according to therules shown in (3) Otherwise the n-th drug is pena-lized ie updated in a way that appears to be less bene-ficial based on the past observations It is not difficult tosee that this algorithm is always more likely to rewardor beneficially update a given drug Since the algorithmproposed in [11] adaptively determines how to updatethe drug concentration based on previous observationsit can also effectively deal with drug response functionsshown in Figures 1A and 1B for which the Gur Gamealgorithm does not perform well Despite its merits thisalgorithm also has its own shortcomings For exampleas the update rule for a given drug is determined onlybased on the two observations that correspond to its lat-est update not on a longer-range trend the algorithmmay be sensitive to small variations in the drugresponse As a result it may not show satisfactorysearch performance for drug response functions that aresimilar to the one in Figure 1C Furthermore consider-ing that f(x) has to be experimentally estimated where acertain level of measurement noise and small variationsdue to a number of practical factors may not be avoid-able such sensitivity may adversely affect the overallperformance of the algorithm Another weakness of thealgorithm is that it only utilizes a very small part of thepast observations without fully utilizing them In the fol-lowing section we introduce a novel stochastic searchalgorithm that can effectively address the aforemen-tioned issues

The adaptive reference update (ARU) stochastic searchalgorithmIn order to make the search algorithm robust to smallvariations in the observed drug response the updaterules have to be decided based on the general trend ofthe drug response change over a wide range of drugconcentration not just based on the response changeresulting from a single-level concentration change Basedon this motivation we propose a novel algorithm calledthe adaptive reference update (ARU) algorithm Inthis algorithm we compare the current drug responsef(xc) with the response f(xref) of a reference drug combi-nation xref which is adaptively updated based on pastobservations In the beginning xref is set to the initialdrug concentration where we start the search processDuring the search when the algorithm encounters alocal maximum the current reference combination isreplaced by the corresponding drug combination As anexample let us consider the one-dimensional drugresponse function f(x) in Figure 2 For illustration weconsider the following hypothetical search processwhere the drug concentration is constantly updatedfrom left to right starting from the lowest concentrationcmin to the highest concentration cmax Suppose thesearch begins at the concentration x = cmin Initially thereference concentration is also set to this initial drugconcentration xref not cmin As the search reaches thefirst local maximum the reference concentration isupdated to this local maximum solution xref not xref1 Asthe search continues to the right this reference concen-tration is used until we reach the next local maximumAfter passing the second local maximum solution xref2the reference point is updated to xref not xref2 In a simi-lar manner as the search continues further to the rightthe reference point is updated to xref not xref3 after pas-sing the third local maximum pointAt each iteration the current drug response f(xc) is

compared to the response f(xref) of the reference drugcombination based on which the drug update rule is

Figure 1 Drug response functions (A) A normalized drug response f(x) that is always below 05 (B) A normalized drug response f(x) that isalways above 05 (C) A drug response function with a large number of small variations

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 4 of 15

determined For example let xc = (middot middot middot xcn middot middot middot ) andxref = (middot middot middot xrefn middot middot middot ) and assume that we want to updatethe concentration of the n-th drug by comparing thetwo drug response values f(xc) and f(xref) As before wehave the following four possible cases

(Case minus 1) xcn lt xrefn and f (xc) lt f (xref) increasing the concentration is more beneficial

(Case minus 2) xcn gt xrefn and f (xc) gt f (xref) increasing the concentration is more beneficial

(Case minus 3) xcn lt xrefn and f (xc) gt f (xref) decreasing the concentration is more beneficial

(Case minus 4) xcn gt xrefn and f (xc) lt f (xref) decreasing the concentration is more beneficial

(5)

Conceptually we can view the above as estimating theldquovirtualrdquo slope between two points (xcn f (x

c)) and(xrefn f (xref)) as follows

f (xref) minus f (xc)xrefn minus xcn

(6)

based on which we determine how to update the con-centration xn of the n-th drug to increase the drugresponse f( x) Given the update rules in (5) the actualupdate decision is made by evaluating the followingfunction

g(xc xref) =12

1 + α middot max

[f (xc) f (xref)

](7)

and comparing it with a random number rn Icirc [0 1] Ifg(xc xref) gtrn the drug concentration xn is updated by asingle level following the beneficial update directionpredicted by (5) Otherwise the concentration isupdated in the opposite direction As briefly mentionedbefore the parameter a Icirc [0 1] controls the random-ness of the algorithm For example a = 0 will make thesearch process completely random regardless of theobserved drug responses Using a larger a means thatwe are giving a larger weight to the past observationswhen deciding how to update the drug concentrationsThe value of this control parameter is limited to a le 1such that g(xc xref) le 1 Also note that we always have g(xc xref) ge 05 which implies that at any drug updatestep the update is always more likely to take place inaccordance with the rules in (5) which have beenderived based on past observations of the drug responseIn other words the ARU algorithm tries to effectively

Figure 2 Updating the reference point As the search for the optimal drug concentration continues from left to right (from the lowestconcentration to the highest one) the reference concentration is updated from the initial drug concentration cmin to the local maximum pointsxref1 xref2 and xref3 according to this order

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 5 of 15

utilize the past response data to beneficially update thedrug concentrations and ultimately to identify a potentdrug combination while keeping the search still stochas-tic For illustration let us again consider the drugresponse function in Figure 2 where the hypotheticalsearch process proceeds from the lowest drug concen-tration to the highest concentration The black solidarrows below the graph shows the drug update directionthat gets higher probability according to the new algo-rithm described above For example in region-A(cmin lt times lt xref1) the algorithm tends to increase thedrug concentration x further as the response f(x) is lar-ger than f(cmin) of the initial reference concentration(ie cmin) As x continues to increase and passes thefirst local maximum point xref1 the reference is updatedto xref not xref1 In region-B (xref1 lt times lt xref2) the searchalgorithm tends to drive the concentration towards xref1

by decreasing the concentration Suppose the searchcontinues to increase the drug concentration x beyondxref2 the second local maximum point despite the ten-dency of the algorithm to decrease x back to xref1 Afterpassing xref2 the reference is updated to xref not xref2 Inregion-C the search algorithms assigns higher probabil-ity to the update rule that tries to bring the concentra-tion down to xref2 since f(x) lt f(xref2) in the givenregion However once x enters region-D where f(x) gtf(xref2) the algorithm begins to drive the drug concen-tration x further to the right until it passes the thirdlocal maximum point xref3 The reference concentra-tion is updated to xref not xref3 once the search con-tinues to the right and the drug concentration x getslarger than xref3 Since f(xref3) is larger than f(x) inregion-D (xref3 lt times lt cmax) the search algorithm willtend to bring the concentration down to the currentreference concentration namely xref = xref3Choosing a local maximum solution as a reference com-

bination has a number of practical advantages First of allit allows the algorithm to adjust the drug update rulesbased on a long-range trend of the given drug responsefunction which makes the algorithm robust to small varia-tions in the observed response Another advantage ofusing a long-range trend is that the search process willbecome also less sensitive to random fluctuations that mayexist in the observed drug response Considering that thedrug response function f(x) has to be experimentally esti-mated through actual biological experiments where ran-dom factors (eg measurement noise) that affect theestimation results cannot be completely ruled out suchrobustness is critical for the algorithm to be used in practi-cal drug optimization applications It is also beneficial touse the drug combination that corresponds to the mostrecent local maximum response instead of the drug com-bination that has yielded the highest response among allpast combinations as the reference point This prevents

the search process from dwelling too much on past obser-vations while keeping it robust to variations and randomfluctuations

Drug response functionsIn order to evaluate the overall performance of the ARUalgorithm we used the algorithm to search for the opti-mal drug cocktail for several different drug responsefunctionsTwo-dimensional drug response functions For

performance assessment we first used the four two-dimensional drug response functions that are shown inFigure 3 The first drug response function f2a(x1 x2)shown in Figure 3A is the normalized HIV inhibitorresponse obtained from [17] where x1 Icirc 0 001 003009 027 082 247 741 2222 6667(nM) was consid-ered for Maraviroc and x2 Icirc 0 009 027 08 241722 2167 65(nM) for ROAb14 The second drugresponse f2b(x1 x2) shown in Figure 3B is the normal-ized second De Jong function (Rosenbrockrsquos saddle)[18] We considered x1 x2 Icirc c0 c1 c20 where ck =4(k20 - 05) obtained by evenly dividing the range [-22] into 21 distinct values The third drug response func-tion f2c(x1 x2) in Figure 3C is the normalized lung can-cer inhibition response obtained from [1] where x1 Icirc0 1 2 4 6 8 12 16 20 22(μM ) was considered forChlorpromazine and x2 Icirc 0 025 04 06 08 1 15 24 68(μM ) for Pentamidine Finally Figure 3D showsthe fourth response function f2d(x1 x2) the normalizedbacterial (S aureus) inhibition response reported in [6]x1 Icirc 0 008 016 032 063 125 25 5 10 was con-sidered for Trimethoprim and x2 Icirc 0 031 062 12525 5 10 20 40 was considered for SulfamethoxazoleAll four drug response functions were normalized tospan the entire range [0 1] such that the minimumresponse is 0 and the maximum response is 1Multi-dimensional drug response functions To eval-

uate the performance for optimizing multi-drug cocktailswe defined several hypothetical drug response functionswith up to six drugs First we defined two 3-dimensionaldrug functions f3a(x1 x2 x3) and f3b(x1 x2 x3) The firstfunction is defined as

f3a(x1 x2 x3) = x21 middot sin2(x2) middot cos2(x3) (8)

where each of x1 x2 x3 can take one of the 11 discreteconcentrations that evenly divide the range [-25 25]The second function is defined as follows

f3b(x1 x2 x3) = peaks(x1 x2) middot x3 (9)

using the Matlab peaks(x1 x2) function We assumethat each drug can take one of the 11 discrete valuesthat evenly divide [-3 3] Next we defined two 4-dimen-sional drug functions

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 6 of 15

f4a(x1 x2 x3 x4) = x1 middot eminus(x21+x22+x

23+x

24) (10)

and

f4b(x1 x2 x3 x4) = cos2(03x1) middot sin(03x2) middot tan(01x3) middot x4 (11)

We assume that x1 and x2 in the first function f4a(x1x2 x3 x4) can take one of the 11 discrete values thatevenly divide the range [-2 2] while x3 and x4 can takeone of the 11 discrete values that evenly divide therange [-3 3] For the second drug response function f4b(x1 x2 x3 x4) we assume that each drug can take oneof the 11 distinct values that evenly divide [-3 3] Inaddition we also defined the following 5-dimensionaldrug response functions

f5a(x1 x2 x3 x4 x5) = eminusx1 middot cos2(x2) middot x23 middot[eminus(x4+2)

2minus(x5+3)2+ eminus(x4minus2)2minus(x5minus3)2

](12)

and

f5b(x1 x2 x3 x4 x5) =12peaks(x1 x2) middot cos(05x3) middot sin(05x4) middot x25 (13)

For the first function f5a(x1 x2 x3 x4 x5) x1 and x2 areallowed to take any value from the set of values obtainedby evenly dividing the range [-2 2] into 11 discrete con-centrations The remaining drug concentrations (x3 x4and x5) can take any of the 11 concentrations that evenlydivide [-45 45] For the second drug response functionf5b(x1 x2 x3 x4 x5) we assume that each drug can takeone of the 11 discrete values that evenly divide [-3 3]Finally we also defined two 6-dimensional drug responsefunctions

f6a(x1 x2 x3 x4 x5 x6) = eminus075(x1) middot (sin2(x2) + cos(x3)

) middot eminus075(x24+x25) middot x6 (14)

and

f6b(x1 x2 x3 x4 x5 x6) = eminus01(x21minusx22)minus01(x23+x24) middot cos2(02x35) middot sin(02x36) (15)

where every drug concentration can take its valuefrom one of the 11 discrete concentrations that evenlydivide the range [-25 25]

Figure 3 Two-dimensional drug response functions (A) Inhibition of HIV (B) Second De Jong function (Rosenbrockrsquos saddle) (C) Inhibition ofA549 lung carcinoma cell proliferation (D) Inhibition of bacteria (S aureus) proliferation

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 7 of 15

ResultsOptimizing the combination of two drugsWe first evaluated the overall performance of theARU stochastic search algorithm based on four two-dimensional drug response functions (see Methods) (i)HIV inhibitor response f2a(x1 x2) (ii) second De Jongfunction (Rosenbrockrsquos saddle) f2b(x1 x2) (iii) normal-ized lung cancer inhibition response f2c(x1 x2) and (iv)bacterial (S aureus) inhibition response f2d(x1 x2)These functions are shown in Figure 3 For each drugresponse function we searched for the optimal drugresponse using the proposed algorithm starting fromrandomly selected drug concentrations x1 and x2 Theparameter a that controls the randomness of the searchwas set to a = 1 which implies that the algorithm adap-tively determines the search direction (ie how toupdate the current drug concentration) by fully utilizingthe past observations Note that setting the parameter toa = 0 in (7) would make the search completely randomand independent of the past observations at each stepthe concentration of a given drug will be randomlyincreased or decreased with equal probability regardlessof the update rules given in (5) In each search experi-ment the iterations were repeated up to two times thetotal number of possible drug combinations Thisexperiment was repeated 5000 times to obtain reliableresults For comparison we performed similar experi-ments using the Gur Game algorithm [9] and the sto-chastic search algorithm proposed in [11] with a = 1We computed the following two performance metricsthe success rate and the average number of unique drugcombinations that need to be tested The first metric isdefined as the relative proportion of experiments inwhich the algorithm was able to identify a potent drugcombination whose response is within 5 of the maxi-mum response ie f(x1 x2) ge 095 The second metricis defined as the average number of unique drug combi-nations that have to be tested until a potent drug com-bination is identified in case the search is successfulThese performance assessment results are summarizedin Table 1 Note that the Gur Game algorithm has beentested using both the simultaneous update strategy as

well as the sequential update strategy Unlike the ARUalgorithm and the search algorithm proposed in [11]which update one drug at a time (ie ldquosequentialrdquo drugupdate) the original Gur Game algorithm adopted in[9] updates all drugs simultaneously However it is alsopossible to use the sequential update strategy with theGur Game algorithm and we have evaluated both stra-tegies for comparison Table 1 shows that the proposedARU algorithm outperforms the existing algorithms interms of success rate Furthermore when the successrates are comparable the ARU algorithm can in generalidentify an effective drug combination more efficientlyas reflected in the smaller number of unique drug com-binations that need to be tested We can get a morecomplete picture of the efficiency of the ARU algorithmfrom Figure S1 and Figure S2 which respectively showthe distribution of the number of unique drug combina-tions that need to be tested and the distribution of thenumber of iterations that are needed to identify a potentdrug combination (see Additional file 1)

Optimizing multi-drug cocktailsNext we tested the performance of the ARU algorithmfor optimizing multi-drug cocktails that consist of threeto six drugs For this purpose we used the eight hypothe-tical drug response functions that were defined before(see Methods) As in our previous experiments for eachdrug response function we used the proposed ARU algo-rithm (with a = 1) to search for a potent drug combina-tion whose response is within 5 of the maximumresponse (ie f(x) ge 095) In each search experiment westarted from an randomly selected initial concentrationsand continued the search up to 1000 steps for 3 drugs2000 steps for 4 drugs 3000 steps for 5 drugs and 4000steps for 6 drugs This experiment was repeated 5000times to obtain reliable performance assessment resultsThe simulation results are summarized in Table 2 Asshown in this table the proposed algorithm boasts a sig-nificantly higher success rate compared to the Gur Gamealgorithm [9] It also results in either comparable orslightly improved success rate compared to the previousdrug optimization algorithm (a = 1) [11] However we

Table 1 Performance for optimizing the combination of two drugs

Gur Game(simultaneous)

Gur Game (sequential) Previous searchalgorithm [11](a = 1)

ARU algorithm(proposed) (a = 1)

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

f2a(x) HIV INHIBITION (M = 80) 97 132 95 172 100 134 100 121

f2b(x) DEJONG (2ND) (M = 441) 9 36 10 45 99 562 99 462

f2c(x) CANCERINHIBITION

(M = 100) 58 113 53 130 98 132 98 124

f2d(x) BACTERIAINHIBITION

(M = 81) 96 59 91 68 100 48 100 45

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 8 of 15

can see that the ARU algorithm clearly outperforms theprevious search algorithm in terms of efficiency asreflected in the significantly smaller number of uniquedrug combinations that need to be tested until an effec-tive combination is identified Figures S3 and S5 showthe distribution of the number of unique drug combina-tions that need to be tested to identify an effective drugcocktail (see Additional file 1) Similarly Figures S4 andS6 show the distribution of the number of search itera-tions that are needed by each algorithm

Drug optimization in the presence of measurement noiseIn order to use a drug optimization algorithm in practi-cal applications the algorithm has to be robust to ran-dom fluctuations in the estimated drug response Toevaluate the robustness of the proposed ARU algorithmwe evaluated its search performance in the presence ofmeasurement noise and compared it with other existingstochastic search algorithms In these experiments weconsidered two different types of search strategies Inthe first search strategy (referred as type-A) when thesearch algorithm happens to revisit a drug combinationthat was previously tested it does not re-evaluate thedrug response and simply uses the previously estimatedvalue On the other hand according to the second strat-egy (referred as type-B) the search algorithm always re-evaluates the drug response even if it revisits a pre-viously evaluated drug combination since the measuredresponse may be different every time due to the randommeasurement noise The first strategy may be usefulwhen the noise level is relatively low in which case thisstrategy may be able to reduce the total number of drugresponse evaluations thereby reducing the overallexperimental cost for identifying a potent drug combi-nation However when the noise level is high thesearch performance may be degraded as the search algo-rithm clings to the past (noisy) response once it hasbeen measured In contrast the second search strategygenerally requires a relatively larger number of drugresponse evaluations but it tends to be more robust to

random fluctuations and noise in the measured drugresponse functionIn order to evaluate the performance of the different

search algorithms in the presence of noise we per-formed similar search experiments as before at three dif-ferent levels of additive noise 2 5 and 8 Moreprecisely we assume that

fobs(x) = ftrue(x) + η

where fobs(x) is the observed drug response ftrue(x) is thetrue response and h is an independent random noise thatis uniformly distributed over (-u u) where u Icirc 002 005008 For each drug response function and a given noiselevel u we tested the performance of both search strate-gies For type-A search we evaluated the success rate andthe average number of unique drug combinations thathave to be tested until a potent drug combination is identi-fied For type-B search we evaluated the success rate andthe average number of iterations instead of the number ofunique drug combinations until an effective combinationis identified This is because in a type-B search the searchalgorithm re-evaluates the drug response even if it revisitsthe same drug combination that was previously tested Thesimulation results are shown in Table 3 4 5 6 7 for drugresponse functions with two to six drugs The parameter awas set to a = 1 for the ARU algorithm as well as the pre-vious search algorithm proposed in [11]As we can see in these Tables measurement noise cer-

tainly affects the overall performance of the ARU algo-rithm where a higher noise tends to reduce the successrate and increase the number of iterations as well as thatof the unique drug combinations to be tested For manydrug response functions considered in our simulationsthe performance degradation is typically not too signifi-cant for the proposed algorithm showing that the ARUalgorithm is relatively robust to measurement noiseHowever we can also observe that the extent of perfor-mance degradation will critically depend on the land-scape of the underlying drug response In most cases the

Table 2 Performance for optimizing multi-drug cocktails

Gur Game (simultaneous) Gur Game (sequential) Previous search algorithm [11](a = 1)

ARU algorithm (proposed)(a = 1)

success rate unique comb success rate unique comb success rate unique comb success rate unique comb

f3a(x) (M = 113) 1 43 2 55 100 1053 100 740

f3b(x) (M = 113) 83 2294 58 2048 100 885 100 794

f4a(x) (M = 114) 20 8239 11 6666 100 1779 100 1368

f4b(x) (M = 114) 52 7067 24 5209 100 1179 100 916

f5a(x) (M = 115) 8 21 2 48 100 1381 100 806

f5b(x) (M = 115) 89 10134 54 9762 100 2529 100 2168

f6a(x) (M = 116) 90 12691 44 12608 100 1919 100 1781

f6b(x) (M = 116) 90 4467 40 10332 100 2381 100 1901

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 9 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f2a(x) (2) A success rateunique comb

97 96 100 100

125 170 133 116

B success rateiterations

97 95 100 100

378 452 258 200

(5) A success rateunique comb

97 96 100 100

126 171 133 118

B success rateiterations

97 95 100 100

380 454 266 202

(8) A success rateunique comb

97 96 100 100

126 170 133 120

B success rateiterations

97 95 100 100

382 454 268 204

f2b(x) (2) A success rateunique comb

10 10 99 99

39 42 570 451

B success rateiterations

10 10 99 99

40 44 1485 1200

(5) A success rateunique comb

9 9 98 98

41 45 629 522

B success rateiterations

9 9 98 99

43 47 1721 1438

(8) A success rateunique comb

8 9 97 98

41 47 662 556

B success rateiterations

9 9 97 98

44 49 1981 1673

f2c(x) (2) A success rateunique comb

61 54 98 98

109 127 131 125

B success rateiterations

60 54 98 98

331 360 359 352

(5) A success rateunique comb

71 66 98 98

114 132 129 124

B success rateiterations

60 54 98 98

332 354 367 360

(8) A success rateunique comb

88 83 98 98

117 134 126 120

B success rateiterations

6 54 98 98

334 343 374 370

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 10 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise (Continued)

f2d(x) (2) A success rateunique comb

100 100 100 100

49 60 46 41

B success rateiterations

96 9 100 100

183 197 82 77

(5) A success rateunique comb

100 100 100 100

49 57 43 41

B success rateiterations

96 91 100 100

184 196 81 75

(8) A success rateunique comb

100 100 100 100

48 56 44 42

B success rateiterations

96 91 100 100

186 196 81 71

Table 4 Performance for optimizing the combination of three drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f3a(x) (2) A success rateunique comb

1 3 99 100

24 73 1106 773

B success rateiterations

1 3 99 100

28 109 2011 1396

(5) A success rateunique comb

1 3 99 100

24 74 1117 784

B success rateiterations

1 3 99 100

25 101 2016 1440

(8) A success rateunique comb

1 3 99 100

25 77 1133 805

B success rateiterations

1 3 99 100

23 94 2109 1517

f3b(x) (2) A success rateunique comb

86 69 99 99

2243 2016 1108 933

B success rateiterations

83 59 99 99

3671 4191 2115 2053

(5) A success rateunique comb

89 72 99 99

2223 2016 1166 1062

B success rateiterations

83 59 98 98

3593 4399 2255 2229

(8) A success rateunique comb

90 74 97 98

2259 2009 1264 1143

B success rateiterations

82 60 97 98

3594 4313 2491 2468

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 11 of 15

Table 5 Performance for optimizing the combination of four drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f4a(x) (2) A success rateunique comb

21 13 96 98

7989 7117 3939 3274

B success rateiterations

21 12 96 97

9419 10321 5583 4523

(5) A success rateunique comb

21 14 90 95

8163 6536 4731 3983

B success rateiterations

21 13 90 95

8956 10229 6754 5812

(8) A success rateunique comb

24 14 85 95

8587 6816 5057 4336

B success rateiterations

23 13 84 92

9971 10089 7208 6485

f4b(x) (2) A success rateunique comb

62 41 100 100

6345 5231 1380 1031

B success rateiterations

51 26 100 100

9324 9030 2369 1828

(5) A success rateunique comb

75 68 100 100

6102 4680 2311 1509

B success rateiterations

50 25 100 100

8559 9212 4110 2588

(8) A success rateunique comb

86 82 98 100

5258 4301 3142 2159

B success rateiterations

50 24 94 100

8353 9792 6020 3938

Table 6 Performance for optimizing the combination of five drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f5a(x) (2) A success rateunique comb

8 9 100 100

21 44 1393 1225

B success rateiterations

9 11 100 100

21 60 1724 1549

(5) A success rateunique comb

9 11 100 100

39 79 1421 1291

B success rateiterations

9 12 100 100

1083 386 1771 1556

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 12 of 15

Table 6 Performance for optimizing the combination of five drugs in the presence of noise (Continued)

(8) A success rateunique comb

10 13 100 100

71 202 1445 1314

B success rateiterations

9 13 100 100

1914 708 1823 1565

f5b(x) (2) A success rateunique comb

89 55 100 100

9179 10263 4071 3439

B success rateiterations

90 55 100 100

9998 13251 5165 4443

(5) A success rateunique comb

90 59 97 98

9328 10027 5077 4636

B success rateiterations

90 56 99 99

10047 13327 6569 5624

(8) A success rateunique comb

91 59 97 98

9595 9712 5788 5348

B success rateiterations

90 56 99 99

10152 13410 7350 6686

Table 7 Performance for optimizing the combination of six drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f6a(x) (2) A success rateunique comb

91 43 100 100

12800 12141 4768 4326

B success rateiterations

90 43 100 100

13526 16626 5314 5036

(5) A success rateunique comb

90 44 99 99

12623 12472 6210 5983

B success rateiterations

90 45 100 100

13968 16759 7637 7361

(8) A success rateunique comb

90 46 98 98

12047 13024 7232 6988

B success rateiterations

90 46 98 98

14122 16819 8752 8343

f6b(x) (2) A success rateunique comb

91 43 100 100

5099 9710 3414 2377

B success rateiterations

94 44 100 100

12407 16460 4365 2935

(5) A success rateunique comb

90 42 100 100

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 13 of 15

ARU algorithm continued to substantially outperformother stochastic search algorithms [911] demonstratingthat it is better suited for practical drug optimizationapplicationsOne interesting observation is that the performance of

the Gur Game algorithm is typically not very sensitiveto measurement noise In fact in some cases its perfor-mance even improves as the noise level goes up Themain reason for this phenomenon is as follows As dis-cussed earlier the Gur Game algorithm does not adaptto the observed drug response function and for this rea-son its overall performance crucially depends onwhether or not its predetermined FSA matches the drugresponse function at hand As a result if the FSA doesnot match the original drug response function well iro-nically enough the measurement noise may perturb thesearch process in such a way that improves the overallperformance In this sense the fact that the Gur Gamealgorithm is not very sensitive to measurement noisereflects its inaptitude for handling various types of drugresponse functions rather than its robustness to randomfluctuations and noise in the measured drug response

ConclusionsIn this paper we proposed a novel stochastic searchalgorithm called the adaptive reference update (ARU)algorithm which can be effectively used for optimizingthe composition of combinatory drugs The proposedalgorithm intelligently utilizes the drug response valuesobserved in the past to reliably predict how to benefi-cially update the drug concentrations to improve thedrug response As we demonstrated throughout thispaper the proposed algorithm addresses several short-comings of previous drug optimization algorithms[911] thereby improving the overall search perfor-mance Numerical experiments based on various typesof multi-drug response functions show that the ARUalgorithm results in a higher success rate (ie higherprobability of identifying a potent drug combination)while requiring significantly fewer drug response evalua-tions Furthermore the proposed algorithm is robust to

random measurement noise where its search perfor-mance is not substantially affected in the presence ofnoise and degrades gracefully as the noise levelincreases Throughout our experiments we used a = 1for the ARU algorithm as well as the previous searchalgorithm [11] As discussed earlier the parameter acontrols the randomness of the search by determininghow much weight we should give to the drug responsevalues that we observed in the past In general unlessthe observations are very noisy or the underlying drugresponse function is assumed to be extremely nonlinearit would be best to set the parameter to the largestallowed value (ie a = 1) so that we fully utilize thepast observations for making our best informed guessabout the beneficial drug update strategy For compari-son we also repeated our simulations using a = 05 anda = 075 whose results are summarized in Table S1 -Table S7 (see Additional file 1) We can see from theseresults that a = 1 indeed leads to the best performancefor the drug response functions and the noise levels wehave considered in this paper

Additional material

Additional file 1 Performance of the ARU algorithm Furtherperformance evaluation results of the proposed Adaptive ReferenceUpdate (ARU) algorithm

AcknowledgementsBased on ldquoEfficient combinatorial drug optimization through stochasticsearchrdquo by Mansuck Kim and Byung-Jun Yoon which appeared in GenomicSignal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop oncopy 2011 IEEE [19]This work was supported in part by the National Science Foundationthrough NSF Award CCF-1149544This article has been published as part of BMC Genomics Volume 13Supplement 6 2012 Selected articles from the IEEE International Workshopon Genomic Signal Processing and Statistics (GENSIPS) 2011 The fullcontents of the supplement are available online at httpwwwbiomedcentralcombmcgenomicssupplements13S6

Authorsrsquo contributionsConceived and developed the algorithm MK BJY Performed theexperiments MK Analyzed the data and wrote the paper MK BJY

Table 7 Performance for optimizing the combination of six drugs in the presence of noise (Continued)

4738 9707 3499 2798

B success rateiterations

94 44 100 100

12786 17049 4808 3246

(8) A success rateunique comb

89 42 100 100

4544 9699 4574 3532

B success rateiterations

94 44 100 100

13331 17755 5453 3915

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 14 of 15

with a normalized drug response of f (xc) The algo-rithm generates N random numbers rn Icirc [01] onefor each drug Each rn is compared against the currentdrug response xc = (xc1 x

c2 middot middot middot xcN) which is used to

either ldquorewardrdquo or ldquopenalizerdquo the n-th drug For exam-ple the drug is rewarded if f (xc) gtrn Otherwise it ispenalized This update process is repeated for each ofthe N drugs According to this scheme drug combina-tions with higher response are more likely to berewarded while combinations with lower response aremore likely to be penalized In the long run the algo-rithm is expected to drive the concentration of eachdrug towards the one that maximizes the responseNow an important relevant question is what is theright way of rewarding (or penalizing) the currentconcentration of a given drug The Gur Game algo-rithm used in [9] uses a predetermined finite stateautomaton (FSA) for this purpose For example letxc = ckn isin Cn be the current concentration of the n-thdrug Suppose we have f (xc) gtrn hence the currentconcentration xcn of the n-th drug should be rewardedThe FSA in [9] is designed such that the drug concen-tration is increased further if the current concentra-tion xcn is larger than a reference concentration crefn and decreased further if xcn is smaller than crefn Morespecifically if xcn = ckn and f(xc) gtrn then the drug con-centration is updated as follows

xcn =

⎧⎪⎪⎨⎪⎪⎩

ck+1n if xcn gt crefn and k lt Mn

ckn if xcn gt crefn and k = Mn

ckminus1n if xcn lt crefn and k gt 1ckn if xcn lt crefn and k = 1

(1)

The reference concentration crefn is typically chosen asthe median of the set Cn As shown in (1) the drugconcentration remains unchanged if it cannot beincreased (or decreased) further Now suppose that f

(xc) ltrn and therefore the current concentration xcn = cknshould be penalized In this case the concentration isupdated in the opposite direction

xcn =ckminus1n if xcn gt crefnck+1n if xcn lt crefn

(2)

Note that penalization moves the current drug con-

centration closer to the reference concentration crefn As

previously discussed in [11] one of the weaknesses ofthe Gur Game algorithm is that it uses a predeterminedFSA for updating (ie rewardingpenalizing) the drugconcentrations and does not adapt to the drug responsefunction at hand which is not known in advance As aresult the algorithm may perform poorly unless thedrug response function f(x) is properly normalized and

the reference concentration crefn is chosen adequately for

each drug For example consider the one-dimensionaldrug response f(x) shown in Figure 1A As we can seethe drug response has been over-normalized hence f(x) lt05 for any allowed concentration x Icirc [cmin cmax] Sincef(x) lt 05 a uniformly distributed random number r Icirc[01] is more likely to be larger than f(x) This impliesthat the Gur Game algorithm always tends to penalizethe current drug concentration (no matter what its valueis) which will probabilistically drive the concentrationtowards cref although it is clearly not optimal Figure 1Bshows another drug response for which the Gur Gamealgorithm will not work properly In this example wehave f(x) gt 05 hence the Gur Game algorithm is alwaysmore likely to reward the current drug concentrationwhich tends to drive the concentration away from thereference concentration cref This will push the concen-tration either towards cmin or cmax both of which aresuboptimalThe enhanced stochastic algorithm proposed in [11]

addresses this problem by making the search algorithmadapt to a given drug response The basic idea of thisalgorithm is to make an ldquoinformed-guessrdquo about how tobeneficially update a given drug concentration insteadof following a predetermined update rule Unlike theGur Game algorithm in [9] where all N drugs aresimultaneously updated based on the (same) currentdrug response f(xc) the enhanced algorithm updates theconcentration of one drug at a time As an examplesuppose during the last update of the n-th drug thedrug combination has been updated as

x = (x1 middot middot middot middot middot middot xN) rArr xprime = (xprime1 middot middot middot middot middot middot xprime

N)

where x and xprime are identical except for the concentra-

tion of the n-th drug hence xi = xprimei(i = n)and

xi = xprimei(i = n) We assume that xn and xprime

n differ only by

a single concentration level so that xn = ckn and

xprimen = ck+1n or xn = ck+1n and xprime

n = ckn for some k The algo-

rithm compares the two drug responses f (x) and f (xrsquo) thereby determine wether it would be more beneficial tofurther increase or decrease the concentration of then-th drug For example we may have the following fourcases

(Case - 1) xn lt xprimen and f (x) lt f (xprime) increasing the concentration is more beneficial

(Case - 2) xn gt xprimen and f (x) gt f (xprime) increasing the concentration is more beneficial

(Case - 3) xn lt xprimen and f (x) gt f (xprime) decreasing the concentration is more beneficial

(Case - 4) xn gt xprimen and f (x) lt f (xprime) decreasing the concentration is more beneficial

(3)

The above rules allow the algorithm to adaptivelydetermine how to reward (or penalize) a given drug con-centration based on the observed drug response valuesHowever the decision whether to reward or penalize the

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 3 of 15

current drug is made in a probabilistic manner For thispurpose we evaluate the following function

g(x xprime) =12

1 + α middot max

[f (x) f (xprime)

] (4)

where a Icirc [0 1] is a control parameter that adjuststhe randomness of the algorithm [11] This g(x xrsquo) iscompared with a uniformly distributed random numberrn Icirc [0 1] If g(x xrsquo) gt rn the n-th drug is rewardedie updated in such a way that appears to be more ben-eficial for enhancing the drug response according to therules shown in (3) Otherwise the n-th drug is pena-lized ie updated in a way that appears to be less bene-ficial based on the past observations It is not difficult tosee that this algorithm is always more likely to rewardor beneficially update a given drug Since the algorithmproposed in [11] adaptively determines how to updatethe drug concentration based on previous observationsit can also effectively deal with drug response functionsshown in Figures 1A and 1B for which the Gur Gamealgorithm does not perform well Despite its merits thisalgorithm also has its own shortcomings For exampleas the update rule for a given drug is determined onlybased on the two observations that correspond to its lat-est update not on a longer-range trend the algorithmmay be sensitive to small variations in the drugresponse As a result it may not show satisfactorysearch performance for drug response functions that aresimilar to the one in Figure 1C Furthermore consider-ing that f(x) has to be experimentally estimated where acertain level of measurement noise and small variationsdue to a number of practical factors may not be avoid-able such sensitivity may adversely affect the overallperformance of the algorithm Another weakness of thealgorithm is that it only utilizes a very small part of thepast observations without fully utilizing them In the fol-lowing section we introduce a novel stochastic searchalgorithm that can effectively address the aforemen-tioned issues

The adaptive reference update (ARU) stochastic searchalgorithmIn order to make the search algorithm robust to smallvariations in the observed drug response the updaterules have to be decided based on the general trend ofthe drug response change over a wide range of drugconcentration not just based on the response changeresulting from a single-level concentration change Basedon this motivation we propose a novel algorithm calledthe adaptive reference update (ARU) algorithm Inthis algorithm we compare the current drug responsef(xc) with the response f(xref) of a reference drug combi-nation xref which is adaptively updated based on pastobservations In the beginning xref is set to the initialdrug concentration where we start the search processDuring the search when the algorithm encounters alocal maximum the current reference combination isreplaced by the corresponding drug combination As anexample let us consider the one-dimensional drugresponse function f(x) in Figure 2 For illustration weconsider the following hypothetical search processwhere the drug concentration is constantly updatedfrom left to right starting from the lowest concentrationcmin to the highest concentration cmax Suppose thesearch begins at the concentration x = cmin Initially thereference concentration is also set to this initial drugconcentration xref not cmin As the search reaches thefirst local maximum the reference concentration isupdated to this local maximum solution xref not xref1 Asthe search continues to the right this reference concen-tration is used until we reach the next local maximumAfter passing the second local maximum solution xref2the reference point is updated to xref not xref2 In a simi-lar manner as the search continues further to the rightthe reference point is updated to xref not xref3 after pas-sing the third local maximum pointAt each iteration the current drug response f(xc) is

compared to the response f(xref) of the reference drugcombination based on which the drug update rule is

Figure 1 Drug response functions (A) A normalized drug response f(x) that is always below 05 (B) A normalized drug response f(x) that isalways above 05 (C) A drug response function with a large number of small variations

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 4 of 15

determined For example let xc = (middot middot middot xcn middot middot middot ) andxref = (middot middot middot xrefn middot middot middot ) and assume that we want to updatethe concentration of the n-th drug by comparing thetwo drug response values f(xc) and f(xref) As before wehave the following four possible cases

(Case minus 1) xcn lt xrefn and f (xc) lt f (xref) increasing the concentration is more beneficial

(Case minus 2) xcn gt xrefn and f (xc) gt f (xref) increasing the concentration is more beneficial

(Case minus 3) xcn lt xrefn and f (xc) gt f (xref) decreasing the concentration is more beneficial

(Case minus 4) xcn gt xrefn and f (xc) lt f (xref) decreasing the concentration is more beneficial

(5)

Conceptually we can view the above as estimating theldquovirtualrdquo slope between two points (xcn f (x

c)) and(xrefn f (xref)) as follows

f (xref) minus f (xc)xrefn minus xcn

(6)

based on which we determine how to update the con-centration xn of the n-th drug to increase the drugresponse f( x) Given the update rules in (5) the actualupdate decision is made by evaluating the followingfunction

g(xc xref) =12

1 + α middot max

[f (xc) f (xref)

](7)

and comparing it with a random number rn Icirc [0 1] Ifg(xc xref) gtrn the drug concentration xn is updated by asingle level following the beneficial update directionpredicted by (5) Otherwise the concentration isupdated in the opposite direction As briefly mentionedbefore the parameter a Icirc [0 1] controls the random-ness of the algorithm For example a = 0 will make thesearch process completely random regardless of theobserved drug responses Using a larger a means thatwe are giving a larger weight to the past observationswhen deciding how to update the drug concentrationsThe value of this control parameter is limited to a le 1such that g(xc xref) le 1 Also note that we always have g(xc xref) ge 05 which implies that at any drug updatestep the update is always more likely to take place inaccordance with the rules in (5) which have beenderived based on past observations of the drug responseIn other words the ARU algorithm tries to effectively

Figure 2 Updating the reference point As the search for the optimal drug concentration continues from left to right (from the lowestconcentration to the highest one) the reference concentration is updated from the initial drug concentration cmin to the local maximum pointsxref1 xref2 and xref3 according to this order

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 5 of 15

utilize the past response data to beneficially update thedrug concentrations and ultimately to identify a potentdrug combination while keeping the search still stochas-tic For illustration let us again consider the drugresponse function in Figure 2 where the hypotheticalsearch process proceeds from the lowest drug concen-tration to the highest concentration The black solidarrows below the graph shows the drug update directionthat gets higher probability according to the new algo-rithm described above For example in region-A(cmin lt times lt xref1) the algorithm tends to increase thedrug concentration x further as the response f(x) is lar-ger than f(cmin) of the initial reference concentration(ie cmin) As x continues to increase and passes thefirst local maximum point xref1 the reference is updatedto xref not xref1 In region-B (xref1 lt times lt xref2) the searchalgorithm tends to drive the concentration towards xref1

by decreasing the concentration Suppose the searchcontinues to increase the drug concentration x beyondxref2 the second local maximum point despite the ten-dency of the algorithm to decrease x back to xref1 Afterpassing xref2 the reference is updated to xref not xref2 Inregion-C the search algorithms assigns higher probabil-ity to the update rule that tries to bring the concentra-tion down to xref2 since f(x) lt f(xref2) in the givenregion However once x enters region-D where f(x) gtf(xref2) the algorithm begins to drive the drug concen-tration x further to the right until it passes the thirdlocal maximum point xref3 The reference concentra-tion is updated to xref not xref3 once the search con-tinues to the right and the drug concentration x getslarger than xref3 Since f(xref3) is larger than f(x) inregion-D (xref3 lt times lt cmax) the search algorithm willtend to bring the concentration down to the currentreference concentration namely xref = xref3Choosing a local maximum solution as a reference com-

bination has a number of practical advantages First of allit allows the algorithm to adjust the drug update rulesbased on a long-range trend of the given drug responsefunction which makes the algorithm robust to small varia-tions in the observed response Another advantage ofusing a long-range trend is that the search process willbecome also less sensitive to random fluctuations that mayexist in the observed drug response Considering that thedrug response function f(x) has to be experimentally esti-mated through actual biological experiments where ran-dom factors (eg measurement noise) that affect theestimation results cannot be completely ruled out suchrobustness is critical for the algorithm to be used in practi-cal drug optimization applications It is also beneficial touse the drug combination that corresponds to the mostrecent local maximum response instead of the drug com-bination that has yielded the highest response among allpast combinations as the reference point This prevents

the search process from dwelling too much on past obser-vations while keeping it robust to variations and randomfluctuations

Drug response functionsIn order to evaluate the overall performance of the ARUalgorithm we used the algorithm to search for the opti-mal drug cocktail for several different drug responsefunctionsTwo-dimensional drug response functions For

performance assessment we first used the four two-dimensional drug response functions that are shown inFigure 3 The first drug response function f2a(x1 x2)shown in Figure 3A is the normalized HIV inhibitorresponse obtained from [17] where x1 Icirc 0 001 003009 027 082 247 741 2222 6667(nM) was consid-ered for Maraviroc and x2 Icirc 0 009 027 08 241722 2167 65(nM) for ROAb14 The second drugresponse f2b(x1 x2) shown in Figure 3B is the normal-ized second De Jong function (Rosenbrockrsquos saddle)[18] We considered x1 x2 Icirc c0 c1 c20 where ck =4(k20 - 05) obtained by evenly dividing the range [-22] into 21 distinct values The third drug response func-tion f2c(x1 x2) in Figure 3C is the normalized lung can-cer inhibition response obtained from [1] where x1 Icirc0 1 2 4 6 8 12 16 20 22(μM ) was considered forChlorpromazine and x2 Icirc 0 025 04 06 08 1 15 24 68(μM ) for Pentamidine Finally Figure 3D showsthe fourth response function f2d(x1 x2) the normalizedbacterial (S aureus) inhibition response reported in [6]x1 Icirc 0 008 016 032 063 125 25 5 10 was con-sidered for Trimethoprim and x2 Icirc 0 031 062 12525 5 10 20 40 was considered for SulfamethoxazoleAll four drug response functions were normalized tospan the entire range [0 1] such that the minimumresponse is 0 and the maximum response is 1Multi-dimensional drug response functions To eval-

uate the performance for optimizing multi-drug cocktailswe defined several hypothetical drug response functionswith up to six drugs First we defined two 3-dimensionaldrug functions f3a(x1 x2 x3) and f3b(x1 x2 x3) The firstfunction is defined as

f3a(x1 x2 x3) = x21 middot sin2(x2) middot cos2(x3) (8)

where each of x1 x2 x3 can take one of the 11 discreteconcentrations that evenly divide the range [-25 25]The second function is defined as follows

f3b(x1 x2 x3) = peaks(x1 x2) middot x3 (9)

using the Matlab peaks(x1 x2) function We assumethat each drug can take one of the 11 discrete valuesthat evenly divide [-3 3] Next we defined two 4-dimen-sional drug functions

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 6 of 15

f4a(x1 x2 x3 x4) = x1 middot eminus(x21+x22+x

23+x

24) (10)

and

f4b(x1 x2 x3 x4) = cos2(03x1) middot sin(03x2) middot tan(01x3) middot x4 (11)

We assume that x1 and x2 in the first function f4a(x1x2 x3 x4) can take one of the 11 discrete values thatevenly divide the range [-2 2] while x3 and x4 can takeone of the 11 discrete values that evenly divide therange [-3 3] For the second drug response function f4b(x1 x2 x3 x4) we assume that each drug can take oneof the 11 distinct values that evenly divide [-3 3] Inaddition we also defined the following 5-dimensionaldrug response functions

f5a(x1 x2 x3 x4 x5) = eminusx1 middot cos2(x2) middot x23 middot[eminus(x4+2)

2minus(x5+3)2+ eminus(x4minus2)2minus(x5minus3)2

](12)

and

f5b(x1 x2 x3 x4 x5) =12peaks(x1 x2) middot cos(05x3) middot sin(05x4) middot x25 (13)

For the first function f5a(x1 x2 x3 x4 x5) x1 and x2 areallowed to take any value from the set of values obtainedby evenly dividing the range [-2 2] into 11 discrete con-centrations The remaining drug concentrations (x3 x4and x5) can take any of the 11 concentrations that evenlydivide [-45 45] For the second drug response functionf5b(x1 x2 x3 x4 x5) we assume that each drug can takeone of the 11 discrete values that evenly divide [-3 3]Finally we also defined two 6-dimensional drug responsefunctions

f6a(x1 x2 x3 x4 x5 x6) = eminus075(x1) middot (sin2(x2) + cos(x3)

) middot eminus075(x24+x25) middot x6 (14)

and

f6b(x1 x2 x3 x4 x5 x6) = eminus01(x21minusx22)minus01(x23+x24) middot cos2(02x35) middot sin(02x36) (15)

where every drug concentration can take its valuefrom one of the 11 discrete concentrations that evenlydivide the range [-25 25]

Figure 3 Two-dimensional drug response functions (A) Inhibition of HIV (B) Second De Jong function (Rosenbrockrsquos saddle) (C) Inhibition ofA549 lung carcinoma cell proliferation (D) Inhibition of bacteria (S aureus) proliferation

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 7 of 15

ResultsOptimizing the combination of two drugsWe first evaluated the overall performance of theARU stochastic search algorithm based on four two-dimensional drug response functions (see Methods) (i)HIV inhibitor response f2a(x1 x2) (ii) second De Jongfunction (Rosenbrockrsquos saddle) f2b(x1 x2) (iii) normal-ized lung cancer inhibition response f2c(x1 x2) and (iv)bacterial (S aureus) inhibition response f2d(x1 x2)These functions are shown in Figure 3 For each drugresponse function we searched for the optimal drugresponse using the proposed algorithm starting fromrandomly selected drug concentrations x1 and x2 Theparameter a that controls the randomness of the searchwas set to a = 1 which implies that the algorithm adap-tively determines the search direction (ie how toupdate the current drug concentration) by fully utilizingthe past observations Note that setting the parameter toa = 0 in (7) would make the search completely randomand independent of the past observations at each stepthe concentration of a given drug will be randomlyincreased or decreased with equal probability regardlessof the update rules given in (5) In each search experi-ment the iterations were repeated up to two times thetotal number of possible drug combinations Thisexperiment was repeated 5000 times to obtain reliableresults For comparison we performed similar experi-ments using the Gur Game algorithm [9] and the sto-chastic search algorithm proposed in [11] with a = 1We computed the following two performance metricsthe success rate and the average number of unique drugcombinations that need to be tested The first metric isdefined as the relative proportion of experiments inwhich the algorithm was able to identify a potent drugcombination whose response is within 5 of the maxi-mum response ie f(x1 x2) ge 095 The second metricis defined as the average number of unique drug combi-nations that have to be tested until a potent drug com-bination is identified in case the search is successfulThese performance assessment results are summarizedin Table 1 Note that the Gur Game algorithm has beentested using both the simultaneous update strategy as

well as the sequential update strategy Unlike the ARUalgorithm and the search algorithm proposed in [11]which update one drug at a time (ie ldquosequentialrdquo drugupdate) the original Gur Game algorithm adopted in[9] updates all drugs simultaneously However it is alsopossible to use the sequential update strategy with theGur Game algorithm and we have evaluated both stra-tegies for comparison Table 1 shows that the proposedARU algorithm outperforms the existing algorithms interms of success rate Furthermore when the successrates are comparable the ARU algorithm can in generalidentify an effective drug combination more efficientlyas reflected in the smaller number of unique drug com-binations that need to be tested We can get a morecomplete picture of the efficiency of the ARU algorithmfrom Figure S1 and Figure S2 which respectively showthe distribution of the number of unique drug combina-tions that need to be tested and the distribution of thenumber of iterations that are needed to identify a potentdrug combination (see Additional file 1)

Optimizing multi-drug cocktailsNext we tested the performance of the ARU algorithmfor optimizing multi-drug cocktails that consist of threeto six drugs For this purpose we used the eight hypothe-tical drug response functions that were defined before(see Methods) As in our previous experiments for eachdrug response function we used the proposed ARU algo-rithm (with a = 1) to search for a potent drug combina-tion whose response is within 5 of the maximumresponse (ie f(x) ge 095) In each search experiment westarted from an randomly selected initial concentrationsand continued the search up to 1000 steps for 3 drugs2000 steps for 4 drugs 3000 steps for 5 drugs and 4000steps for 6 drugs This experiment was repeated 5000times to obtain reliable performance assessment resultsThe simulation results are summarized in Table 2 Asshown in this table the proposed algorithm boasts a sig-nificantly higher success rate compared to the Gur Gamealgorithm [9] It also results in either comparable orslightly improved success rate compared to the previousdrug optimization algorithm (a = 1) [11] However we

Table 1 Performance for optimizing the combination of two drugs

Gur Game(simultaneous)

Gur Game (sequential) Previous searchalgorithm [11](a = 1)

ARU algorithm(proposed) (a = 1)

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

f2a(x) HIV INHIBITION (M = 80) 97 132 95 172 100 134 100 121

f2b(x) DEJONG (2ND) (M = 441) 9 36 10 45 99 562 99 462

f2c(x) CANCERINHIBITION

(M = 100) 58 113 53 130 98 132 98 124

f2d(x) BACTERIAINHIBITION

(M = 81) 96 59 91 68 100 48 100 45

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 8 of 15

can see that the ARU algorithm clearly outperforms theprevious search algorithm in terms of efficiency asreflected in the significantly smaller number of uniquedrug combinations that need to be tested until an effec-tive combination is identified Figures S3 and S5 showthe distribution of the number of unique drug combina-tions that need to be tested to identify an effective drugcocktail (see Additional file 1) Similarly Figures S4 andS6 show the distribution of the number of search itera-tions that are needed by each algorithm

Drug optimization in the presence of measurement noiseIn order to use a drug optimization algorithm in practi-cal applications the algorithm has to be robust to ran-dom fluctuations in the estimated drug response Toevaluate the robustness of the proposed ARU algorithmwe evaluated its search performance in the presence ofmeasurement noise and compared it with other existingstochastic search algorithms In these experiments weconsidered two different types of search strategies Inthe first search strategy (referred as type-A) when thesearch algorithm happens to revisit a drug combinationthat was previously tested it does not re-evaluate thedrug response and simply uses the previously estimatedvalue On the other hand according to the second strat-egy (referred as type-B) the search algorithm always re-evaluates the drug response even if it revisits a pre-viously evaluated drug combination since the measuredresponse may be different every time due to the randommeasurement noise The first strategy may be usefulwhen the noise level is relatively low in which case thisstrategy may be able to reduce the total number of drugresponse evaluations thereby reducing the overallexperimental cost for identifying a potent drug combi-nation However when the noise level is high thesearch performance may be degraded as the search algo-rithm clings to the past (noisy) response once it hasbeen measured In contrast the second search strategygenerally requires a relatively larger number of drugresponse evaluations but it tends to be more robust to

random fluctuations and noise in the measured drugresponse functionIn order to evaluate the performance of the different

search algorithms in the presence of noise we per-formed similar search experiments as before at three dif-ferent levels of additive noise 2 5 and 8 Moreprecisely we assume that

fobs(x) = ftrue(x) + η

where fobs(x) is the observed drug response ftrue(x) is thetrue response and h is an independent random noise thatis uniformly distributed over (-u u) where u Icirc 002 005008 For each drug response function and a given noiselevel u we tested the performance of both search strate-gies For type-A search we evaluated the success rate andthe average number of unique drug combinations thathave to be tested until a potent drug combination is identi-fied For type-B search we evaluated the success rate andthe average number of iterations instead of the number ofunique drug combinations until an effective combinationis identified This is because in a type-B search the searchalgorithm re-evaluates the drug response even if it revisitsthe same drug combination that was previously tested Thesimulation results are shown in Table 3 4 5 6 7 for drugresponse functions with two to six drugs The parameter awas set to a = 1 for the ARU algorithm as well as the pre-vious search algorithm proposed in [11]As we can see in these Tables measurement noise cer-

tainly affects the overall performance of the ARU algo-rithm where a higher noise tends to reduce the successrate and increase the number of iterations as well as thatof the unique drug combinations to be tested For manydrug response functions considered in our simulationsthe performance degradation is typically not too signifi-cant for the proposed algorithm showing that the ARUalgorithm is relatively robust to measurement noiseHowever we can also observe that the extent of perfor-mance degradation will critically depend on the land-scape of the underlying drug response In most cases the

Table 2 Performance for optimizing multi-drug cocktails

Gur Game (simultaneous) Gur Game (sequential) Previous search algorithm [11](a = 1)

ARU algorithm (proposed)(a = 1)

success rate unique comb success rate unique comb success rate unique comb success rate unique comb

f3a(x) (M = 113) 1 43 2 55 100 1053 100 740

f3b(x) (M = 113) 83 2294 58 2048 100 885 100 794

f4a(x) (M = 114) 20 8239 11 6666 100 1779 100 1368

f4b(x) (M = 114) 52 7067 24 5209 100 1179 100 916

f5a(x) (M = 115) 8 21 2 48 100 1381 100 806

f5b(x) (M = 115) 89 10134 54 9762 100 2529 100 2168

f6a(x) (M = 116) 90 12691 44 12608 100 1919 100 1781

f6b(x) (M = 116) 90 4467 40 10332 100 2381 100 1901

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 9 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f2a(x) (2) A success rateunique comb

97 96 100 100

125 170 133 116

B success rateiterations

97 95 100 100

378 452 258 200

(5) A success rateunique comb

97 96 100 100

126 171 133 118

B success rateiterations

97 95 100 100

380 454 266 202

(8) A success rateunique comb

97 96 100 100

126 170 133 120

B success rateiterations

97 95 100 100

382 454 268 204

f2b(x) (2) A success rateunique comb

10 10 99 99

39 42 570 451

B success rateiterations

10 10 99 99

40 44 1485 1200

(5) A success rateunique comb

9 9 98 98

41 45 629 522

B success rateiterations

9 9 98 99

43 47 1721 1438

(8) A success rateunique comb

8 9 97 98

41 47 662 556

B success rateiterations

9 9 97 98

44 49 1981 1673

f2c(x) (2) A success rateunique comb

61 54 98 98

109 127 131 125

B success rateiterations

60 54 98 98

331 360 359 352

(5) A success rateunique comb

71 66 98 98

114 132 129 124

B success rateiterations

60 54 98 98

332 354 367 360

(8) A success rateunique comb

88 83 98 98

117 134 126 120

B success rateiterations

6 54 98 98

334 343 374 370

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 10 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise (Continued)

f2d(x) (2) A success rateunique comb

100 100 100 100

49 60 46 41

B success rateiterations

96 9 100 100

183 197 82 77

(5) A success rateunique comb

100 100 100 100

49 57 43 41

B success rateiterations

96 91 100 100

184 196 81 75

(8) A success rateunique comb

100 100 100 100

48 56 44 42

B success rateiterations

96 91 100 100

186 196 81 71

Table 4 Performance for optimizing the combination of three drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f3a(x) (2) A success rateunique comb

1 3 99 100

24 73 1106 773

B success rateiterations

1 3 99 100

28 109 2011 1396

(5) A success rateunique comb

1 3 99 100

24 74 1117 784

B success rateiterations

1 3 99 100

25 101 2016 1440

(8) A success rateunique comb

1 3 99 100

25 77 1133 805

B success rateiterations

1 3 99 100

23 94 2109 1517

f3b(x) (2) A success rateunique comb

86 69 99 99

2243 2016 1108 933

B success rateiterations

83 59 99 99

3671 4191 2115 2053

(5) A success rateunique comb

89 72 99 99

2223 2016 1166 1062

B success rateiterations

83 59 98 98

3593 4399 2255 2229

(8) A success rateunique comb

90 74 97 98

2259 2009 1264 1143

B success rateiterations

82 60 97 98

3594 4313 2491 2468

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 11 of 15

Table 5 Performance for optimizing the combination of four drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f4a(x) (2) A success rateunique comb

21 13 96 98

7989 7117 3939 3274

B success rateiterations

21 12 96 97

9419 10321 5583 4523

(5) A success rateunique comb

21 14 90 95

8163 6536 4731 3983

B success rateiterations

21 13 90 95

8956 10229 6754 5812

(8) A success rateunique comb

24 14 85 95

8587 6816 5057 4336

B success rateiterations

23 13 84 92

9971 10089 7208 6485

f4b(x) (2) A success rateunique comb

62 41 100 100

6345 5231 1380 1031

B success rateiterations

51 26 100 100

9324 9030 2369 1828

(5) A success rateunique comb

75 68 100 100

6102 4680 2311 1509

B success rateiterations

50 25 100 100

8559 9212 4110 2588

(8) A success rateunique comb

86 82 98 100

5258 4301 3142 2159

B success rateiterations

50 24 94 100

8353 9792 6020 3938

Table 6 Performance for optimizing the combination of five drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f5a(x) (2) A success rateunique comb

8 9 100 100

21 44 1393 1225

B success rateiterations

9 11 100 100

21 60 1724 1549

(5) A success rateunique comb

9 11 100 100

39 79 1421 1291

B success rateiterations

9 12 100 100

1083 386 1771 1556

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 12 of 15

Table 6 Performance for optimizing the combination of five drugs in the presence of noise (Continued)

(8) A success rateunique comb

10 13 100 100

71 202 1445 1314

B success rateiterations

9 13 100 100

1914 708 1823 1565

f5b(x) (2) A success rateunique comb

89 55 100 100

9179 10263 4071 3439

B success rateiterations

90 55 100 100

9998 13251 5165 4443

(5) A success rateunique comb

90 59 97 98

9328 10027 5077 4636

B success rateiterations

90 56 99 99

10047 13327 6569 5624

(8) A success rateunique comb

91 59 97 98

9595 9712 5788 5348

B success rateiterations

90 56 99 99

10152 13410 7350 6686

Table 7 Performance for optimizing the combination of six drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f6a(x) (2) A success rateunique comb

91 43 100 100

12800 12141 4768 4326

B success rateiterations

90 43 100 100

13526 16626 5314 5036

(5) A success rateunique comb

90 44 99 99

12623 12472 6210 5983

B success rateiterations

90 45 100 100

13968 16759 7637 7361

(8) A success rateunique comb

90 46 98 98

12047 13024 7232 6988

B success rateiterations

90 46 98 98

14122 16819 8752 8343

f6b(x) (2) A success rateunique comb

91 43 100 100

5099 9710 3414 2377

B success rateiterations

94 44 100 100

12407 16460 4365 2935

(5) A success rateunique comb

90 42 100 100

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 13 of 15

ARU algorithm continued to substantially outperformother stochastic search algorithms [911] demonstratingthat it is better suited for practical drug optimizationapplicationsOne interesting observation is that the performance of

the Gur Game algorithm is typically not very sensitiveto measurement noise In fact in some cases its perfor-mance even improves as the noise level goes up Themain reason for this phenomenon is as follows As dis-cussed earlier the Gur Game algorithm does not adaptto the observed drug response function and for this rea-son its overall performance crucially depends onwhether or not its predetermined FSA matches the drugresponse function at hand As a result if the FSA doesnot match the original drug response function well iro-nically enough the measurement noise may perturb thesearch process in such a way that improves the overallperformance In this sense the fact that the Gur Gamealgorithm is not very sensitive to measurement noisereflects its inaptitude for handling various types of drugresponse functions rather than its robustness to randomfluctuations and noise in the measured drug response

ConclusionsIn this paper we proposed a novel stochastic searchalgorithm called the adaptive reference update (ARU)algorithm which can be effectively used for optimizingthe composition of combinatory drugs The proposedalgorithm intelligently utilizes the drug response valuesobserved in the past to reliably predict how to benefi-cially update the drug concentrations to improve thedrug response As we demonstrated throughout thispaper the proposed algorithm addresses several short-comings of previous drug optimization algorithms[911] thereby improving the overall search perfor-mance Numerical experiments based on various typesof multi-drug response functions show that the ARUalgorithm results in a higher success rate (ie higherprobability of identifying a potent drug combination)while requiring significantly fewer drug response evalua-tions Furthermore the proposed algorithm is robust to

random measurement noise where its search perfor-mance is not substantially affected in the presence ofnoise and degrades gracefully as the noise levelincreases Throughout our experiments we used a = 1for the ARU algorithm as well as the previous searchalgorithm [11] As discussed earlier the parameter acontrols the randomness of the search by determininghow much weight we should give to the drug responsevalues that we observed in the past In general unlessthe observations are very noisy or the underlying drugresponse function is assumed to be extremely nonlinearit would be best to set the parameter to the largestallowed value (ie a = 1) so that we fully utilize thepast observations for making our best informed guessabout the beneficial drug update strategy For compari-son we also repeated our simulations using a = 05 anda = 075 whose results are summarized in Table S1 -Table S7 (see Additional file 1) We can see from theseresults that a = 1 indeed leads to the best performancefor the drug response functions and the noise levels wehave considered in this paper

Additional material

Additional file 1 Performance of the ARU algorithm Furtherperformance evaluation results of the proposed Adaptive ReferenceUpdate (ARU) algorithm

AcknowledgementsBased on ldquoEfficient combinatorial drug optimization through stochasticsearchrdquo by Mansuck Kim and Byung-Jun Yoon which appeared in GenomicSignal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop oncopy 2011 IEEE [19]This work was supported in part by the National Science Foundationthrough NSF Award CCF-1149544This article has been published as part of BMC Genomics Volume 13Supplement 6 2012 Selected articles from the IEEE International Workshopon Genomic Signal Processing and Statistics (GENSIPS) 2011 The fullcontents of the supplement are available online at httpwwwbiomedcentralcombmcgenomicssupplements13S6

Authorsrsquo contributionsConceived and developed the algorithm MK BJY Performed theexperiments MK Analyzed the data and wrote the paper MK BJY

Table 7 Performance for optimizing the combination of six drugs in the presence of noise (Continued)

4738 9707 3499 2798

B success rateiterations

94 44 100 100

12786 17049 4808 3246

(8) A success rateunique comb

89 42 100 100

4544 9699 4574 3532

B success rateiterations

94 44 100 100

13331 17755 5453 3915

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 14 of 15

current drug is made in a probabilistic manner For thispurpose we evaluate the following function

g(x xprime) =12

1 + α middot max

[f (x) f (xprime)

] (4)

where a Icirc [0 1] is a control parameter that adjuststhe randomness of the algorithm [11] This g(x xrsquo) iscompared with a uniformly distributed random numberrn Icirc [0 1] If g(x xrsquo) gt rn the n-th drug is rewardedie updated in such a way that appears to be more ben-eficial for enhancing the drug response according to therules shown in (3) Otherwise the n-th drug is pena-lized ie updated in a way that appears to be less bene-ficial based on the past observations It is not difficult tosee that this algorithm is always more likely to rewardor beneficially update a given drug Since the algorithmproposed in [11] adaptively determines how to updatethe drug concentration based on previous observationsit can also effectively deal with drug response functionsshown in Figures 1A and 1B for which the Gur Gamealgorithm does not perform well Despite its merits thisalgorithm also has its own shortcomings For exampleas the update rule for a given drug is determined onlybased on the two observations that correspond to its lat-est update not on a longer-range trend the algorithmmay be sensitive to small variations in the drugresponse As a result it may not show satisfactorysearch performance for drug response functions that aresimilar to the one in Figure 1C Furthermore consider-ing that f(x) has to be experimentally estimated where acertain level of measurement noise and small variationsdue to a number of practical factors may not be avoid-able such sensitivity may adversely affect the overallperformance of the algorithm Another weakness of thealgorithm is that it only utilizes a very small part of thepast observations without fully utilizing them In the fol-lowing section we introduce a novel stochastic searchalgorithm that can effectively address the aforemen-tioned issues

The adaptive reference update (ARU) stochastic searchalgorithmIn order to make the search algorithm robust to smallvariations in the observed drug response the updaterules have to be decided based on the general trend ofthe drug response change over a wide range of drugconcentration not just based on the response changeresulting from a single-level concentration change Basedon this motivation we propose a novel algorithm calledthe adaptive reference update (ARU) algorithm Inthis algorithm we compare the current drug responsef(xc) with the response f(xref) of a reference drug combi-nation xref which is adaptively updated based on pastobservations In the beginning xref is set to the initialdrug concentration where we start the search processDuring the search when the algorithm encounters alocal maximum the current reference combination isreplaced by the corresponding drug combination As anexample let us consider the one-dimensional drugresponse function f(x) in Figure 2 For illustration weconsider the following hypothetical search processwhere the drug concentration is constantly updatedfrom left to right starting from the lowest concentrationcmin to the highest concentration cmax Suppose thesearch begins at the concentration x = cmin Initially thereference concentration is also set to this initial drugconcentration xref not cmin As the search reaches thefirst local maximum the reference concentration isupdated to this local maximum solution xref not xref1 Asthe search continues to the right this reference concen-tration is used until we reach the next local maximumAfter passing the second local maximum solution xref2the reference point is updated to xref not xref2 In a simi-lar manner as the search continues further to the rightthe reference point is updated to xref not xref3 after pas-sing the third local maximum pointAt each iteration the current drug response f(xc) is

compared to the response f(xref) of the reference drugcombination based on which the drug update rule is

Figure 1 Drug response functions (A) A normalized drug response f(x) that is always below 05 (B) A normalized drug response f(x) that isalways above 05 (C) A drug response function with a large number of small variations

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 4 of 15

determined For example let xc = (middot middot middot xcn middot middot middot ) andxref = (middot middot middot xrefn middot middot middot ) and assume that we want to updatethe concentration of the n-th drug by comparing thetwo drug response values f(xc) and f(xref) As before wehave the following four possible cases

(Case minus 1) xcn lt xrefn and f (xc) lt f (xref) increasing the concentration is more beneficial

(Case minus 2) xcn gt xrefn and f (xc) gt f (xref) increasing the concentration is more beneficial

(Case minus 3) xcn lt xrefn and f (xc) gt f (xref) decreasing the concentration is more beneficial

(Case minus 4) xcn gt xrefn and f (xc) lt f (xref) decreasing the concentration is more beneficial

(5)

Conceptually we can view the above as estimating theldquovirtualrdquo slope between two points (xcn f (x

c)) and(xrefn f (xref)) as follows

f (xref) minus f (xc)xrefn minus xcn

(6)

based on which we determine how to update the con-centration xn of the n-th drug to increase the drugresponse f( x) Given the update rules in (5) the actualupdate decision is made by evaluating the followingfunction

g(xc xref) =12

1 + α middot max

[f (xc) f (xref)

](7)

and comparing it with a random number rn Icirc [0 1] Ifg(xc xref) gtrn the drug concentration xn is updated by asingle level following the beneficial update directionpredicted by (5) Otherwise the concentration isupdated in the opposite direction As briefly mentionedbefore the parameter a Icirc [0 1] controls the random-ness of the algorithm For example a = 0 will make thesearch process completely random regardless of theobserved drug responses Using a larger a means thatwe are giving a larger weight to the past observationswhen deciding how to update the drug concentrationsThe value of this control parameter is limited to a le 1such that g(xc xref) le 1 Also note that we always have g(xc xref) ge 05 which implies that at any drug updatestep the update is always more likely to take place inaccordance with the rules in (5) which have beenderived based on past observations of the drug responseIn other words the ARU algorithm tries to effectively

Figure 2 Updating the reference point As the search for the optimal drug concentration continues from left to right (from the lowestconcentration to the highest one) the reference concentration is updated from the initial drug concentration cmin to the local maximum pointsxref1 xref2 and xref3 according to this order

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 5 of 15

utilize the past response data to beneficially update thedrug concentrations and ultimately to identify a potentdrug combination while keeping the search still stochas-tic For illustration let us again consider the drugresponse function in Figure 2 where the hypotheticalsearch process proceeds from the lowest drug concen-tration to the highest concentration The black solidarrows below the graph shows the drug update directionthat gets higher probability according to the new algo-rithm described above For example in region-A(cmin lt times lt xref1) the algorithm tends to increase thedrug concentration x further as the response f(x) is lar-ger than f(cmin) of the initial reference concentration(ie cmin) As x continues to increase and passes thefirst local maximum point xref1 the reference is updatedto xref not xref1 In region-B (xref1 lt times lt xref2) the searchalgorithm tends to drive the concentration towards xref1

by decreasing the concentration Suppose the searchcontinues to increase the drug concentration x beyondxref2 the second local maximum point despite the ten-dency of the algorithm to decrease x back to xref1 Afterpassing xref2 the reference is updated to xref not xref2 Inregion-C the search algorithms assigns higher probabil-ity to the update rule that tries to bring the concentra-tion down to xref2 since f(x) lt f(xref2) in the givenregion However once x enters region-D where f(x) gtf(xref2) the algorithm begins to drive the drug concen-tration x further to the right until it passes the thirdlocal maximum point xref3 The reference concentra-tion is updated to xref not xref3 once the search con-tinues to the right and the drug concentration x getslarger than xref3 Since f(xref3) is larger than f(x) inregion-D (xref3 lt times lt cmax) the search algorithm willtend to bring the concentration down to the currentreference concentration namely xref = xref3Choosing a local maximum solution as a reference com-

bination has a number of practical advantages First of allit allows the algorithm to adjust the drug update rulesbased on a long-range trend of the given drug responsefunction which makes the algorithm robust to small varia-tions in the observed response Another advantage ofusing a long-range trend is that the search process willbecome also less sensitive to random fluctuations that mayexist in the observed drug response Considering that thedrug response function f(x) has to be experimentally esti-mated through actual biological experiments where ran-dom factors (eg measurement noise) that affect theestimation results cannot be completely ruled out suchrobustness is critical for the algorithm to be used in practi-cal drug optimization applications It is also beneficial touse the drug combination that corresponds to the mostrecent local maximum response instead of the drug com-bination that has yielded the highest response among allpast combinations as the reference point This prevents

the search process from dwelling too much on past obser-vations while keeping it robust to variations and randomfluctuations

Drug response functionsIn order to evaluate the overall performance of the ARUalgorithm we used the algorithm to search for the opti-mal drug cocktail for several different drug responsefunctionsTwo-dimensional drug response functions For

performance assessment we first used the four two-dimensional drug response functions that are shown inFigure 3 The first drug response function f2a(x1 x2)shown in Figure 3A is the normalized HIV inhibitorresponse obtained from [17] where x1 Icirc 0 001 003009 027 082 247 741 2222 6667(nM) was consid-ered for Maraviroc and x2 Icirc 0 009 027 08 241722 2167 65(nM) for ROAb14 The second drugresponse f2b(x1 x2) shown in Figure 3B is the normal-ized second De Jong function (Rosenbrockrsquos saddle)[18] We considered x1 x2 Icirc c0 c1 c20 where ck =4(k20 - 05) obtained by evenly dividing the range [-22] into 21 distinct values The third drug response func-tion f2c(x1 x2) in Figure 3C is the normalized lung can-cer inhibition response obtained from [1] where x1 Icirc0 1 2 4 6 8 12 16 20 22(μM ) was considered forChlorpromazine and x2 Icirc 0 025 04 06 08 1 15 24 68(μM ) for Pentamidine Finally Figure 3D showsthe fourth response function f2d(x1 x2) the normalizedbacterial (S aureus) inhibition response reported in [6]x1 Icirc 0 008 016 032 063 125 25 5 10 was con-sidered for Trimethoprim and x2 Icirc 0 031 062 12525 5 10 20 40 was considered for SulfamethoxazoleAll four drug response functions were normalized tospan the entire range [0 1] such that the minimumresponse is 0 and the maximum response is 1Multi-dimensional drug response functions To eval-

uate the performance for optimizing multi-drug cocktailswe defined several hypothetical drug response functionswith up to six drugs First we defined two 3-dimensionaldrug functions f3a(x1 x2 x3) and f3b(x1 x2 x3) The firstfunction is defined as

f3a(x1 x2 x3) = x21 middot sin2(x2) middot cos2(x3) (8)

where each of x1 x2 x3 can take one of the 11 discreteconcentrations that evenly divide the range [-25 25]The second function is defined as follows

f3b(x1 x2 x3) = peaks(x1 x2) middot x3 (9)

using the Matlab peaks(x1 x2) function We assumethat each drug can take one of the 11 discrete valuesthat evenly divide [-3 3] Next we defined two 4-dimen-sional drug functions

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 6 of 15

f4a(x1 x2 x3 x4) = x1 middot eminus(x21+x22+x

23+x

24) (10)

and

f4b(x1 x2 x3 x4) = cos2(03x1) middot sin(03x2) middot tan(01x3) middot x4 (11)

We assume that x1 and x2 in the first function f4a(x1x2 x3 x4) can take one of the 11 discrete values thatevenly divide the range [-2 2] while x3 and x4 can takeone of the 11 discrete values that evenly divide therange [-3 3] For the second drug response function f4b(x1 x2 x3 x4) we assume that each drug can take oneof the 11 distinct values that evenly divide [-3 3] Inaddition we also defined the following 5-dimensionaldrug response functions

f5a(x1 x2 x3 x4 x5) = eminusx1 middot cos2(x2) middot x23 middot[eminus(x4+2)

2minus(x5+3)2+ eminus(x4minus2)2minus(x5minus3)2

](12)

and

f5b(x1 x2 x3 x4 x5) =12peaks(x1 x2) middot cos(05x3) middot sin(05x4) middot x25 (13)

For the first function f5a(x1 x2 x3 x4 x5) x1 and x2 areallowed to take any value from the set of values obtainedby evenly dividing the range [-2 2] into 11 discrete con-centrations The remaining drug concentrations (x3 x4and x5) can take any of the 11 concentrations that evenlydivide [-45 45] For the second drug response functionf5b(x1 x2 x3 x4 x5) we assume that each drug can takeone of the 11 discrete values that evenly divide [-3 3]Finally we also defined two 6-dimensional drug responsefunctions

f6a(x1 x2 x3 x4 x5 x6) = eminus075(x1) middot (sin2(x2) + cos(x3)

) middot eminus075(x24+x25) middot x6 (14)

and

f6b(x1 x2 x3 x4 x5 x6) = eminus01(x21minusx22)minus01(x23+x24) middot cos2(02x35) middot sin(02x36) (15)

where every drug concentration can take its valuefrom one of the 11 discrete concentrations that evenlydivide the range [-25 25]

Figure 3 Two-dimensional drug response functions (A) Inhibition of HIV (B) Second De Jong function (Rosenbrockrsquos saddle) (C) Inhibition ofA549 lung carcinoma cell proliferation (D) Inhibition of bacteria (S aureus) proliferation

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 7 of 15

ResultsOptimizing the combination of two drugsWe first evaluated the overall performance of theARU stochastic search algorithm based on four two-dimensional drug response functions (see Methods) (i)HIV inhibitor response f2a(x1 x2) (ii) second De Jongfunction (Rosenbrockrsquos saddle) f2b(x1 x2) (iii) normal-ized lung cancer inhibition response f2c(x1 x2) and (iv)bacterial (S aureus) inhibition response f2d(x1 x2)These functions are shown in Figure 3 For each drugresponse function we searched for the optimal drugresponse using the proposed algorithm starting fromrandomly selected drug concentrations x1 and x2 Theparameter a that controls the randomness of the searchwas set to a = 1 which implies that the algorithm adap-tively determines the search direction (ie how toupdate the current drug concentration) by fully utilizingthe past observations Note that setting the parameter toa = 0 in (7) would make the search completely randomand independent of the past observations at each stepthe concentration of a given drug will be randomlyincreased or decreased with equal probability regardlessof the update rules given in (5) In each search experi-ment the iterations were repeated up to two times thetotal number of possible drug combinations Thisexperiment was repeated 5000 times to obtain reliableresults For comparison we performed similar experi-ments using the Gur Game algorithm [9] and the sto-chastic search algorithm proposed in [11] with a = 1We computed the following two performance metricsthe success rate and the average number of unique drugcombinations that need to be tested The first metric isdefined as the relative proportion of experiments inwhich the algorithm was able to identify a potent drugcombination whose response is within 5 of the maxi-mum response ie f(x1 x2) ge 095 The second metricis defined as the average number of unique drug combi-nations that have to be tested until a potent drug com-bination is identified in case the search is successfulThese performance assessment results are summarizedin Table 1 Note that the Gur Game algorithm has beentested using both the simultaneous update strategy as

well as the sequential update strategy Unlike the ARUalgorithm and the search algorithm proposed in [11]which update one drug at a time (ie ldquosequentialrdquo drugupdate) the original Gur Game algorithm adopted in[9] updates all drugs simultaneously However it is alsopossible to use the sequential update strategy with theGur Game algorithm and we have evaluated both stra-tegies for comparison Table 1 shows that the proposedARU algorithm outperforms the existing algorithms interms of success rate Furthermore when the successrates are comparable the ARU algorithm can in generalidentify an effective drug combination more efficientlyas reflected in the smaller number of unique drug com-binations that need to be tested We can get a morecomplete picture of the efficiency of the ARU algorithmfrom Figure S1 and Figure S2 which respectively showthe distribution of the number of unique drug combina-tions that need to be tested and the distribution of thenumber of iterations that are needed to identify a potentdrug combination (see Additional file 1)

Optimizing multi-drug cocktailsNext we tested the performance of the ARU algorithmfor optimizing multi-drug cocktails that consist of threeto six drugs For this purpose we used the eight hypothe-tical drug response functions that were defined before(see Methods) As in our previous experiments for eachdrug response function we used the proposed ARU algo-rithm (with a = 1) to search for a potent drug combina-tion whose response is within 5 of the maximumresponse (ie f(x) ge 095) In each search experiment westarted from an randomly selected initial concentrationsand continued the search up to 1000 steps for 3 drugs2000 steps for 4 drugs 3000 steps for 5 drugs and 4000steps for 6 drugs This experiment was repeated 5000times to obtain reliable performance assessment resultsThe simulation results are summarized in Table 2 Asshown in this table the proposed algorithm boasts a sig-nificantly higher success rate compared to the Gur Gamealgorithm [9] It also results in either comparable orslightly improved success rate compared to the previousdrug optimization algorithm (a = 1) [11] However we

Table 1 Performance for optimizing the combination of two drugs

Gur Game(simultaneous)

Gur Game (sequential) Previous searchalgorithm [11](a = 1)

ARU algorithm(proposed) (a = 1)

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

f2a(x) HIV INHIBITION (M = 80) 97 132 95 172 100 134 100 121

f2b(x) DEJONG (2ND) (M = 441) 9 36 10 45 99 562 99 462

f2c(x) CANCERINHIBITION

(M = 100) 58 113 53 130 98 132 98 124

f2d(x) BACTERIAINHIBITION

(M = 81) 96 59 91 68 100 48 100 45

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 8 of 15

can see that the ARU algorithm clearly outperforms theprevious search algorithm in terms of efficiency asreflected in the significantly smaller number of uniquedrug combinations that need to be tested until an effec-tive combination is identified Figures S3 and S5 showthe distribution of the number of unique drug combina-tions that need to be tested to identify an effective drugcocktail (see Additional file 1) Similarly Figures S4 andS6 show the distribution of the number of search itera-tions that are needed by each algorithm

Drug optimization in the presence of measurement noiseIn order to use a drug optimization algorithm in practi-cal applications the algorithm has to be robust to ran-dom fluctuations in the estimated drug response Toevaluate the robustness of the proposed ARU algorithmwe evaluated its search performance in the presence ofmeasurement noise and compared it with other existingstochastic search algorithms In these experiments weconsidered two different types of search strategies Inthe first search strategy (referred as type-A) when thesearch algorithm happens to revisit a drug combinationthat was previously tested it does not re-evaluate thedrug response and simply uses the previously estimatedvalue On the other hand according to the second strat-egy (referred as type-B) the search algorithm always re-evaluates the drug response even if it revisits a pre-viously evaluated drug combination since the measuredresponse may be different every time due to the randommeasurement noise The first strategy may be usefulwhen the noise level is relatively low in which case thisstrategy may be able to reduce the total number of drugresponse evaluations thereby reducing the overallexperimental cost for identifying a potent drug combi-nation However when the noise level is high thesearch performance may be degraded as the search algo-rithm clings to the past (noisy) response once it hasbeen measured In contrast the second search strategygenerally requires a relatively larger number of drugresponse evaluations but it tends to be more robust to

random fluctuations and noise in the measured drugresponse functionIn order to evaluate the performance of the different

search algorithms in the presence of noise we per-formed similar search experiments as before at three dif-ferent levels of additive noise 2 5 and 8 Moreprecisely we assume that

fobs(x) = ftrue(x) + η

where fobs(x) is the observed drug response ftrue(x) is thetrue response and h is an independent random noise thatis uniformly distributed over (-u u) where u Icirc 002 005008 For each drug response function and a given noiselevel u we tested the performance of both search strate-gies For type-A search we evaluated the success rate andthe average number of unique drug combinations thathave to be tested until a potent drug combination is identi-fied For type-B search we evaluated the success rate andthe average number of iterations instead of the number ofunique drug combinations until an effective combinationis identified This is because in a type-B search the searchalgorithm re-evaluates the drug response even if it revisitsthe same drug combination that was previously tested Thesimulation results are shown in Table 3 4 5 6 7 for drugresponse functions with two to six drugs The parameter awas set to a = 1 for the ARU algorithm as well as the pre-vious search algorithm proposed in [11]As we can see in these Tables measurement noise cer-

tainly affects the overall performance of the ARU algo-rithm where a higher noise tends to reduce the successrate and increase the number of iterations as well as thatof the unique drug combinations to be tested For manydrug response functions considered in our simulationsthe performance degradation is typically not too signifi-cant for the proposed algorithm showing that the ARUalgorithm is relatively robust to measurement noiseHowever we can also observe that the extent of perfor-mance degradation will critically depend on the land-scape of the underlying drug response In most cases the

Table 2 Performance for optimizing multi-drug cocktails

Gur Game (simultaneous) Gur Game (sequential) Previous search algorithm [11](a = 1)

ARU algorithm (proposed)(a = 1)

success rate unique comb success rate unique comb success rate unique comb success rate unique comb

f3a(x) (M = 113) 1 43 2 55 100 1053 100 740

f3b(x) (M = 113) 83 2294 58 2048 100 885 100 794

f4a(x) (M = 114) 20 8239 11 6666 100 1779 100 1368

f4b(x) (M = 114) 52 7067 24 5209 100 1179 100 916

f5a(x) (M = 115) 8 21 2 48 100 1381 100 806

f5b(x) (M = 115) 89 10134 54 9762 100 2529 100 2168

f6a(x) (M = 116) 90 12691 44 12608 100 1919 100 1781

f6b(x) (M = 116) 90 4467 40 10332 100 2381 100 1901

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 9 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f2a(x) (2) A success rateunique comb

97 96 100 100

125 170 133 116

B success rateiterations

97 95 100 100

378 452 258 200

(5) A success rateunique comb

97 96 100 100

126 171 133 118

B success rateiterations

97 95 100 100

380 454 266 202

(8) A success rateunique comb

97 96 100 100

126 170 133 120

B success rateiterations

97 95 100 100

382 454 268 204

f2b(x) (2) A success rateunique comb

10 10 99 99

39 42 570 451

B success rateiterations

10 10 99 99

40 44 1485 1200

(5) A success rateunique comb

9 9 98 98

41 45 629 522

B success rateiterations

9 9 98 99

43 47 1721 1438

(8) A success rateunique comb

8 9 97 98

41 47 662 556

B success rateiterations

9 9 97 98

44 49 1981 1673

f2c(x) (2) A success rateunique comb

61 54 98 98

109 127 131 125

B success rateiterations

60 54 98 98

331 360 359 352

(5) A success rateunique comb

71 66 98 98

114 132 129 124

B success rateiterations

60 54 98 98

332 354 367 360

(8) A success rateunique comb

88 83 98 98

117 134 126 120

B success rateiterations

6 54 98 98

334 343 374 370

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 10 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise (Continued)

f2d(x) (2) A success rateunique comb

100 100 100 100

49 60 46 41

B success rateiterations

96 9 100 100

183 197 82 77

(5) A success rateunique comb

100 100 100 100

49 57 43 41

B success rateiterations

96 91 100 100

184 196 81 75

(8) A success rateunique comb

100 100 100 100

48 56 44 42

B success rateiterations

96 91 100 100

186 196 81 71

Table 4 Performance for optimizing the combination of three drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f3a(x) (2) A success rateunique comb

1 3 99 100

24 73 1106 773

B success rateiterations

1 3 99 100

28 109 2011 1396

(5) A success rateunique comb

1 3 99 100

24 74 1117 784

B success rateiterations

1 3 99 100

25 101 2016 1440

(8) A success rateunique comb

1 3 99 100

25 77 1133 805

B success rateiterations

1 3 99 100

23 94 2109 1517

f3b(x) (2) A success rateunique comb

86 69 99 99

2243 2016 1108 933

B success rateiterations

83 59 99 99

3671 4191 2115 2053

(5) A success rateunique comb

89 72 99 99

2223 2016 1166 1062

B success rateiterations

83 59 98 98

3593 4399 2255 2229

(8) A success rateunique comb

90 74 97 98

2259 2009 1264 1143

B success rateiterations

82 60 97 98

3594 4313 2491 2468

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 11 of 15

Table 5 Performance for optimizing the combination of four drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f4a(x) (2) A success rateunique comb

21 13 96 98

7989 7117 3939 3274

B success rateiterations

21 12 96 97

9419 10321 5583 4523

(5) A success rateunique comb

21 14 90 95

8163 6536 4731 3983

B success rateiterations

21 13 90 95

8956 10229 6754 5812

(8) A success rateunique comb

24 14 85 95

8587 6816 5057 4336

B success rateiterations

23 13 84 92

9971 10089 7208 6485

f4b(x) (2) A success rateunique comb

62 41 100 100

6345 5231 1380 1031

B success rateiterations

51 26 100 100

9324 9030 2369 1828

(5) A success rateunique comb

75 68 100 100

6102 4680 2311 1509

B success rateiterations

50 25 100 100

8559 9212 4110 2588

(8) A success rateunique comb

86 82 98 100

5258 4301 3142 2159

B success rateiterations

50 24 94 100

8353 9792 6020 3938

Table 6 Performance for optimizing the combination of five drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f5a(x) (2) A success rateunique comb

8 9 100 100

21 44 1393 1225

B success rateiterations

9 11 100 100

21 60 1724 1549

(5) A success rateunique comb

9 11 100 100

39 79 1421 1291

B success rateiterations

9 12 100 100

1083 386 1771 1556

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 12 of 15

Table 6 Performance for optimizing the combination of five drugs in the presence of noise (Continued)

(8) A success rateunique comb

10 13 100 100

71 202 1445 1314

B success rateiterations

9 13 100 100

1914 708 1823 1565

f5b(x) (2) A success rateunique comb

89 55 100 100

9179 10263 4071 3439

B success rateiterations

90 55 100 100

9998 13251 5165 4443

(5) A success rateunique comb

90 59 97 98

9328 10027 5077 4636

B success rateiterations

90 56 99 99

10047 13327 6569 5624

(8) A success rateunique comb

91 59 97 98

9595 9712 5788 5348

B success rateiterations

90 56 99 99

10152 13410 7350 6686

Table 7 Performance for optimizing the combination of six drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f6a(x) (2) A success rateunique comb

91 43 100 100

12800 12141 4768 4326

B success rateiterations

90 43 100 100

13526 16626 5314 5036

(5) A success rateunique comb

90 44 99 99

12623 12472 6210 5983

B success rateiterations

90 45 100 100

13968 16759 7637 7361

(8) A success rateunique comb

90 46 98 98

12047 13024 7232 6988

B success rateiterations

90 46 98 98

14122 16819 8752 8343

f6b(x) (2) A success rateunique comb

91 43 100 100

5099 9710 3414 2377

B success rateiterations

94 44 100 100

12407 16460 4365 2935

(5) A success rateunique comb

90 42 100 100

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 13 of 15

ARU algorithm continued to substantially outperformother stochastic search algorithms [911] demonstratingthat it is better suited for practical drug optimizationapplicationsOne interesting observation is that the performance of

the Gur Game algorithm is typically not very sensitiveto measurement noise In fact in some cases its perfor-mance even improves as the noise level goes up Themain reason for this phenomenon is as follows As dis-cussed earlier the Gur Game algorithm does not adaptto the observed drug response function and for this rea-son its overall performance crucially depends onwhether or not its predetermined FSA matches the drugresponse function at hand As a result if the FSA doesnot match the original drug response function well iro-nically enough the measurement noise may perturb thesearch process in such a way that improves the overallperformance In this sense the fact that the Gur Gamealgorithm is not very sensitive to measurement noisereflects its inaptitude for handling various types of drugresponse functions rather than its robustness to randomfluctuations and noise in the measured drug response

ConclusionsIn this paper we proposed a novel stochastic searchalgorithm called the adaptive reference update (ARU)algorithm which can be effectively used for optimizingthe composition of combinatory drugs The proposedalgorithm intelligently utilizes the drug response valuesobserved in the past to reliably predict how to benefi-cially update the drug concentrations to improve thedrug response As we demonstrated throughout thispaper the proposed algorithm addresses several short-comings of previous drug optimization algorithms[911] thereby improving the overall search perfor-mance Numerical experiments based on various typesof multi-drug response functions show that the ARUalgorithm results in a higher success rate (ie higherprobability of identifying a potent drug combination)while requiring significantly fewer drug response evalua-tions Furthermore the proposed algorithm is robust to

random measurement noise where its search perfor-mance is not substantially affected in the presence ofnoise and degrades gracefully as the noise levelincreases Throughout our experiments we used a = 1for the ARU algorithm as well as the previous searchalgorithm [11] As discussed earlier the parameter acontrols the randomness of the search by determininghow much weight we should give to the drug responsevalues that we observed in the past In general unlessthe observations are very noisy or the underlying drugresponse function is assumed to be extremely nonlinearit would be best to set the parameter to the largestallowed value (ie a = 1) so that we fully utilize thepast observations for making our best informed guessabout the beneficial drug update strategy For compari-son we also repeated our simulations using a = 05 anda = 075 whose results are summarized in Table S1 -Table S7 (see Additional file 1) We can see from theseresults that a = 1 indeed leads to the best performancefor the drug response functions and the noise levels wehave considered in this paper

Additional material

Additional file 1 Performance of the ARU algorithm Furtherperformance evaluation results of the proposed Adaptive ReferenceUpdate (ARU) algorithm

AcknowledgementsBased on ldquoEfficient combinatorial drug optimization through stochasticsearchrdquo by Mansuck Kim and Byung-Jun Yoon which appeared in GenomicSignal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop oncopy 2011 IEEE [19]This work was supported in part by the National Science Foundationthrough NSF Award CCF-1149544This article has been published as part of BMC Genomics Volume 13Supplement 6 2012 Selected articles from the IEEE International Workshopon Genomic Signal Processing and Statistics (GENSIPS) 2011 The fullcontents of the supplement are available online at httpwwwbiomedcentralcombmcgenomicssupplements13S6

Authorsrsquo contributionsConceived and developed the algorithm MK BJY Performed theexperiments MK Analyzed the data and wrote the paper MK BJY

Table 7 Performance for optimizing the combination of six drugs in the presence of noise (Continued)

4738 9707 3499 2798

B success rateiterations

94 44 100 100

12786 17049 4808 3246

(8) A success rateunique comb

89 42 100 100

4544 9699 4574 3532

B success rateiterations

94 44 100 100

13331 17755 5453 3915

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 14 of 15

determined For example let xc = (middot middot middot xcn middot middot middot ) andxref = (middot middot middot xrefn middot middot middot ) and assume that we want to updatethe concentration of the n-th drug by comparing thetwo drug response values f(xc) and f(xref) As before wehave the following four possible cases

(Case minus 1) xcn lt xrefn and f (xc) lt f (xref) increasing the concentration is more beneficial

(Case minus 2) xcn gt xrefn and f (xc) gt f (xref) increasing the concentration is more beneficial

(Case minus 3) xcn lt xrefn and f (xc) gt f (xref) decreasing the concentration is more beneficial

(Case minus 4) xcn gt xrefn and f (xc) lt f (xref) decreasing the concentration is more beneficial

(5)

Conceptually we can view the above as estimating theldquovirtualrdquo slope between two points (xcn f (x

c)) and(xrefn f (xref)) as follows

f (xref) minus f (xc)xrefn minus xcn

(6)

based on which we determine how to update the con-centration xn of the n-th drug to increase the drugresponse f( x) Given the update rules in (5) the actualupdate decision is made by evaluating the followingfunction

g(xc xref) =12

1 + α middot max

[f (xc) f (xref)

](7)

and comparing it with a random number rn Icirc [0 1] Ifg(xc xref) gtrn the drug concentration xn is updated by asingle level following the beneficial update directionpredicted by (5) Otherwise the concentration isupdated in the opposite direction As briefly mentionedbefore the parameter a Icirc [0 1] controls the random-ness of the algorithm For example a = 0 will make thesearch process completely random regardless of theobserved drug responses Using a larger a means thatwe are giving a larger weight to the past observationswhen deciding how to update the drug concentrationsThe value of this control parameter is limited to a le 1such that g(xc xref) le 1 Also note that we always have g(xc xref) ge 05 which implies that at any drug updatestep the update is always more likely to take place inaccordance with the rules in (5) which have beenderived based on past observations of the drug responseIn other words the ARU algorithm tries to effectively

Figure 2 Updating the reference point As the search for the optimal drug concentration continues from left to right (from the lowestconcentration to the highest one) the reference concentration is updated from the initial drug concentration cmin to the local maximum pointsxref1 xref2 and xref3 according to this order

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 5 of 15

utilize the past response data to beneficially update thedrug concentrations and ultimately to identify a potentdrug combination while keeping the search still stochas-tic For illustration let us again consider the drugresponse function in Figure 2 where the hypotheticalsearch process proceeds from the lowest drug concen-tration to the highest concentration The black solidarrows below the graph shows the drug update directionthat gets higher probability according to the new algo-rithm described above For example in region-A(cmin lt times lt xref1) the algorithm tends to increase thedrug concentration x further as the response f(x) is lar-ger than f(cmin) of the initial reference concentration(ie cmin) As x continues to increase and passes thefirst local maximum point xref1 the reference is updatedto xref not xref1 In region-B (xref1 lt times lt xref2) the searchalgorithm tends to drive the concentration towards xref1

by decreasing the concentration Suppose the searchcontinues to increase the drug concentration x beyondxref2 the second local maximum point despite the ten-dency of the algorithm to decrease x back to xref1 Afterpassing xref2 the reference is updated to xref not xref2 Inregion-C the search algorithms assigns higher probabil-ity to the update rule that tries to bring the concentra-tion down to xref2 since f(x) lt f(xref2) in the givenregion However once x enters region-D where f(x) gtf(xref2) the algorithm begins to drive the drug concen-tration x further to the right until it passes the thirdlocal maximum point xref3 The reference concentra-tion is updated to xref not xref3 once the search con-tinues to the right and the drug concentration x getslarger than xref3 Since f(xref3) is larger than f(x) inregion-D (xref3 lt times lt cmax) the search algorithm willtend to bring the concentration down to the currentreference concentration namely xref = xref3Choosing a local maximum solution as a reference com-

bination has a number of practical advantages First of allit allows the algorithm to adjust the drug update rulesbased on a long-range trend of the given drug responsefunction which makes the algorithm robust to small varia-tions in the observed response Another advantage ofusing a long-range trend is that the search process willbecome also less sensitive to random fluctuations that mayexist in the observed drug response Considering that thedrug response function f(x) has to be experimentally esti-mated through actual biological experiments where ran-dom factors (eg measurement noise) that affect theestimation results cannot be completely ruled out suchrobustness is critical for the algorithm to be used in practi-cal drug optimization applications It is also beneficial touse the drug combination that corresponds to the mostrecent local maximum response instead of the drug com-bination that has yielded the highest response among allpast combinations as the reference point This prevents

the search process from dwelling too much on past obser-vations while keeping it robust to variations and randomfluctuations

Drug response functionsIn order to evaluate the overall performance of the ARUalgorithm we used the algorithm to search for the opti-mal drug cocktail for several different drug responsefunctionsTwo-dimensional drug response functions For

performance assessment we first used the four two-dimensional drug response functions that are shown inFigure 3 The first drug response function f2a(x1 x2)shown in Figure 3A is the normalized HIV inhibitorresponse obtained from [17] where x1 Icirc 0 001 003009 027 082 247 741 2222 6667(nM) was consid-ered for Maraviroc and x2 Icirc 0 009 027 08 241722 2167 65(nM) for ROAb14 The second drugresponse f2b(x1 x2) shown in Figure 3B is the normal-ized second De Jong function (Rosenbrockrsquos saddle)[18] We considered x1 x2 Icirc c0 c1 c20 where ck =4(k20 - 05) obtained by evenly dividing the range [-22] into 21 distinct values The third drug response func-tion f2c(x1 x2) in Figure 3C is the normalized lung can-cer inhibition response obtained from [1] where x1 Icirc0 1 2 4 6 8 12 16 20 22(μM ) was considered forChlorpromazine and x2 Icirc 0 025 04 06 08 1 15 24 68(μM ) for Pentamidine Finally Figure 3D showsthe fourth response function f2d(x1 x2) the normalizedbacterial (S aureus) inhibition response reported in [6]x1 Icirc 0 008 016 032 063 125 25 5 10 was con-sidered for Trimethoprim and x2 Icirc 0 031 062 12525 5 10 20 40 was considered for SulfamethoxazoleAll four drug response functions were normalized tospan the entire range [0 1] such that the minimumresponse is 0 and the maximum response is 1Multi-dimensional drug response functions To eval-

uate the performance for optimizing multi-drug cocktailswe defined several hypothetical drug response functionswith up to six drugs First we defined two 3-dimensionaldrug functions f3a(x1 x2 x3) and f3b(x1 x2 x3) The firstfunction is defined as

f3a(x1 x2 x3) = x21 middot sin2(x2) middot cos2(x3) (8)

where each of x1 x2 x3 can take one of the 11 discreteconcentrations that evenly divide the range [-25 25]The second function is defined as follows

f3b(x1 x2 x3) = peaks(x1 x2) middot x3 (9)

using the Matlab peaks(x1 x2) function We assumethat each drug can take one of the 11 discrete valuesthat evenly divide [-3 3] Next we defined two 4-dimen-sional drug functions

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 6 of 15

f4a(x1 x2 x3 x4) = x1 middot eminus(x21+x22+x

23+x

24) (10)

and

f4b(x1 x2 x3 x4) = cos2(03x1) middot sin(03x2) middot tan(01x3) middot x4 (11)

We assume that x1 and x2 in the first function f4a(x1x2 x3 x4) can take one of the 11 discrete values thatevenly divide the range [-2 2] while x3 and x4 can takeone of the 11 discrete values that evenly divide therange [-3 3] For the second drug response function f4b(x1 x2 x3 x4) we assume that each drug can take oneof the 11 distinct values that evenly divide [-3 3] Inaddition we also defined the following 5-dimensionaldrug response functions

f5a(x1 x2 x3 x4 x5) = eminusx1 middot cos2(x2) middot x23 middot[eminus(x4+2)

2minus(x5+3)2+ eminus(x4minus2)2minus(x5minus3)2

](12)

and

f5b(x1 x2 x3 x4 x5) =12peaks(x1 x2) middot cos(05x3) middot sin(05x4) middot x25 (13)

For the first function f5a(x1 x2 x3 x4 x5) x1 and x2 areallowed to take any value from the set of values obtainedby evenly dividing the range [-2 2] into 11 discrete con-centrations The remaining drug concentrations (x3 x4and x5) can take any of the 11 concentrations that evenlydivide [-45 45] For the second drug response functionf5b(x1 x2 x3 x4 x5) we assume that each drug can takeone of the 11 discrete values that evenly divide [-3 3]Finally we also defined two 6-dimensional drug responsefunctions

f6a(x1 x2 x3 x4 x5 x6) = eminus075(x1) middot (sin2(x2) + cos(x3)

) middot eminus075(x24+x25) middot x6 (14)

and

f6b(x1 x2 x3 x4 x5 x6) = eminus01(x21minusx22)minus01(x23+x24) middot cos2(02x35) middot sin(02x36) (15)

where every drug concentration can take its valuefrom one of the 11 discrete concentrations that evenlydivide the range [-25 25]

Figure 3 Two-dimensional drug response functions (A) Inhibition of HIV (B) Second De Jong function (Rosenbrockrsquos saddle) (C) Inhibition ofA549 lung carcinoma cell proliferation (D) Inhibition of bacteria (S aureus) proliferation

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 7 of 15

ResultsOptimizing the combination of two drugsWe first evaluated the overall performance of theARU stochastic search algorithm based on four two-dimensional drug response functions (see Methods) (i)HIV inhibitor response f2a(x1 x2) (ii) second De Jongfunction (Rosenbrockrsquos saddle) f2b(x1 x2) (iii) normal-ized lung cancer inhibition response f2c(x1 x2) and (iv)bacterial (S aureus) inhibition response f2d(x1 x2)These functions are shown in Figure 3 For each drugresponse function we searched for the optimal drugresponse using the proposed algorithm starting fromrandomly selected drug concentrations x1 and x2 Theparameter a that controls the randomness of the searchwas set to a = 1 which implies that the algorithm adap-tively determines the search direction (ie how toupdate the current drug concentration) by fully utilizingthe past observations Note that setting the parameter toa = 0 in (7) would make the search completely randomand independent of the past observations at each stepthe concentration of a given drug will be randomlyincreased or decreased with equal probability regardlessof the update rules given in (5) In each search experi-ment the iterations were repeated up to two times thetotal number of possible drug combinations Thisexperiment was repeated 5000 times to obtain reliableresults For comparison we performed similar experi-ments using the Gur Game algorithm [9] and the sto-chastic search algorithm proposed in [11] with a = 1We computed the following two performance metricsthe success rate and the average number of unique drugcombinations that need to be tested The first metric isdefined as the relative proportion of experiments inwhich the algorithm was able to identify a potent drugcombination whose response is within 5 of the maxi-mum response ie f(x1 x2) ge 095 The second metricis defined as the average number of unique drug combi-nations that have to be tested until a potent drug com-bination is identified in case the search is successfulThese performance assessment results are summarizedin Table 1 Note that the Gur Game algorithm has beentested using both the simultaneous update strategy as

well as the sequential update strategy Unlike the ARUalgorithm and the search algorithm proposed in [11]which update one drug at a time (ie ldquosequentialrdquo drugupdate) the original Gur Game algorithm adopted in[9] updates all drugs simultaneously However it is alsopossible to use the sequential update strategy with theGur Game algorithm and we have evaluated both stra-tegies for comparison Table 1 shows that the proposedARU algorithm outperforms the existing algorithms interms of success rate Furthermore when the successrates are comparable the ARU algorithm can in generalidentify an effective drug combination more efficientlyas reflected in the smaller number of unique drug com-binations that need to be tested We can get a morecomplete picture of the efficiency of the ARU algorithmfrom Figure S1 and Figure S2 which respectively showthe distribution of the number of unique drug combina-tions that need to be tested and the distribution of thenumber of iterations that are needed to identify a potentdrug combination (see Additional file 1)

Optimizing multi-drug cocktailsNext we tested the performance of the ARU algorithmfor optimizing multi-drug cocktails that consist of threeto six drugs For this purpose we used the eight hypothe-tical drug response functions that were defined before(see Methods) As in our previous experiments for eachdrug response function we used the proposed ARU algo-rithm (with a = 1) to search for a potent drug combina-tion whose response is within 5 of the maximumresponse (ie f(x) ge 095) In each search experiment westarted from an randomly selected initial concentrationsand continued the search up to 1000 steps for 3 drugs2000 steps for 4 drugs 3000 steps for 5 drugs and 4000steps for 6 drugs This experiment was repeated 5000times to obtain reliable performance assessment resultsThe simulation results are summarized in Table 2 Asshown in this table the proposed algorithm boasts a sig-nificantly higher success rate compared to the Gur Gamealgorithm [9] It also results in either comparable orslightly improved success rate compared to the previousdrug optimization algorithm (a = 1) [11] However we

Table 1 Performance for optimizing the combination of two drugs

Gur Game(simultaneous)

Gur Game (sequential) Previous searchalgorithm [11](a = 1)

ARU algorithm(proposed) (a = 1)

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

f2a(x) HIV INHIBITION (M = 80) 97 132 95 172 100 134 100 121

f2b(x) DEJONG (2ND) (M = 441) 9 36 10 45 99 562 99 462

f2c(x) CANCERINHIBITION

(M = 100) 58 113 53 130 98 132 98 124

f2d(x) BACTERIAINHIBITION

(M = 81) 96 59 91 68 100 48 100 45

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 8 of 15

can see that the ARU algorithm clearly outperforms theprevious search algorithm in terms of efficiency asreflected in the significantly smaller number of uniquedrug combinations that need to be tested until an effec-tive combination is identified Figures S3 and S5 showthe distribution of the number of unique drug combina-tions that need to be tested to identify an effective drugcocktail (see Additional file 1) Similarly Figures S4 andS6 show the distribution of the number of search itera-tions that are needed by each algorithm

Drug optimization in the presence of measurement noiseIn order to use a drug optimization algorithm in practi-cal applications the algorithm has to be robust to ran-dom fluctuations in the estimated drug response Toevaluate the robustness of the proposed ARU algorithmwe evaluated its search performance in the presence ofmeasurement noise and compared it with other existingstochastic search algorithms In these experiments weconsidered two different types of search strategies Inthe first search strategy (referred as type-A) when thesearch algorithm happens to revisit a drug combinationthat was previously tested it does not re-evaluate thedrug response and simply uses the previously estimatedvalue On the other hand according to the second strat-egy (referred as type-B) the search algorithm always re-evaluates the drug response even if it revisits a pre-viously evaluated drug combination since the measuredresponse may be different every time due to the randommeasurement noise The first strategy may be usefulwhen the noise level is relatively low in which case thisstrategy may be able to reduce the total number of drugresponse evaluations thereby reducing the overallexperimental cost for identifying a potent drug combi-nation However when the noise level is high thesearch performance may be degraded as the search algo-rithm clings to the past (noisy) response once it hasbeen measured In contrast the second search strategygenerally requires a relatively larger number of drugresponse evaluations but it tends to be more robust to

random fluctuations and noise in the measured drugresponse functionIn order to evaluate the performance of the different

search algorithms in the presence of noise we per-formed similar search experiments as before at three dif-ferent levels of additive noise 2 5 and 8 Moreprecisely we assume that

fobs(x) = ftrue(x) + η

where fobs(x) is the observed drug response ftrue(x) is thetrue response and h is an independent random noise thatis uniformly distributed over (-u u) where u Icirc 002 005008 For each drug response function and a given noiselevel u we tested the performance of both search strate-gies For type-A search we evaluated the success rate andthe average number of unique drug combinations thathave to be tested until a potent drug combination is identi-fied For type-B search we evaluated the success rate andthe average number of iterations instead of the number ofunique drug combinations until an effective combinationis identified This is because in a type-B search the searchalgorithm re-evaluates the drug response even if it revisitsthe same drug combination that was previously tested Thesimulation results are shown in Table 3 4 5 6 7 for drugresponse functions with two to six drugs The parameter awas set to a = 1 for the ARU algorithm as well as the pre-vious search algorithm proposed in [11]As we can see in these Tables measurement noise cer-

tainly affects the overall performance of the ARU algo-rithm where a higher noise tends to reduce the successrate and increase the number of iterations as well as thatof the unique drug combinations to be tested For manydrug response functions considered in our simulationsthe performance degradation is typically not too signifi-cant for the proposed algorithm showing that the ARUalgorithm is relatively robust to measurement noiseHowever we can also observe that the extent of perfor-mance degradation will critically depend on the land-scape of the underlying drug response In most cases the

Table 2 Performance for optimizing multi-drug cocktails

Gur Game (simultaneous) Gur Game (sequential) Previous search algorithm [11](a = 1)

ARU algorithm (proposed)(a = 1)

success rate unique comb success rate unique comb success rate unique comb success rate unique comb

f3a(x) (M = 113) 1 43 2 55 100 1053 100 740

f3b(x) (M = 113) 83 2294 58 2048 100 885 100 794

f4a(x) (M = 114) 20 8239 11 6666 100 1779 100 1368

f4b(x) (M = 114) 52 7067 24 5209 100 1179 100 916

f5a(x) (M = 115) 8 21 2 48 100 1381 100 806

f5b(x) (M = 115) 89 10134 54 9762 100 2529 100 2168

f6a(x) (M = 116) 90 12691 44 12608 100 1919 100 1781

f6b(x) (M = 116) 90 4467 40 10332 100 2381 100 1901

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 9 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f2a(x) (2) A success rateunique comb

97 96 100 100

125 170 133 116

B success rateiterations

97 95 100 100

378 452 258 200

(5) A success rateunique comb

97 96 100 100

126 171 133 118

B success rateiterations

97 95 100 100

380 454 266 202

(8) A success rateunique comb

97 96 100 100

126 170 133 120

B success rateiterations

97 95 100 100

382 454 268 204

f2b(x) (2) A success rateunique comb

10 10 99 99

39 42 570 451

B success rateiterations

10 10 99 99

40 44 1485 1200

(5) A success rateunique comb

9 9 98 98

41 45 629 522

B success rateiterations

9 9 98 99

43 47 1721 1438

(8) A success rateunique comb

8 9 97 98

41 47 662 556

B success rateiterations

9 9 97 98

44 49 1981 1673

f2c(x) (2) A success rateunique comb

61 54 98 98

109 127 131 125

B success rateiterations

60 54 98 98

331 360 359 352

(5) A success rateunique comb

71 66 98 98

114 132 129 124

B success rateiterations

60 54 98 98

332 354 367 360

(8) A success rateunique comb

88 83 98 98

117 134 126 120

B success rateiterations

6 54 98 98

334 343 374 370

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 10 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise (Continued)

f2d(x) (2) A success rateunique comb

100 100 100 100

49 60 46 41

B success rateiterations

96 9 100 100

183 197 82 77

(5) A success rateunique comb

100 100 100 100

49 57 43 41

B success rateiterations

96 91 100 100

184 196 81 75

(8) A success rateunique comb

100 100 100 100

48 56 44 42

B success rateiterations

96 91 100 100

186 196 81 71

Table 4 Performance for optimizing the combination of three drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f3a(x) (2) A success rateunique comb

1 3 99 100

24 73 1106 773

B success rateiterations

1 3 99 100

28 109 2011 1396

(5) A success rateunique comb

1 3 99 100

24 74 1117 784

B success rateiterations

1 3 99 100

25 101 2016 1440

(8) A success rateunique comb

1 3 99 100

25 77 1133 805

B success rateiterations

1 3 99 100

23 94 2109 1517

f3b(x) (2) A success rateunique comb

86 69 99 99

2243 2016 1108 933

B success rateiterations

83 59 99 99

3671 4191 2115 2053

(5) A success rateunique comb

89 72 99 99

2223 2016 1166 1062

B success rateiterations

83 59 98 98

3593 4399 2255 2229

(8) A success rateunique comb

90 74 97 98

2259 2009 1264 1143

B success rateiterations

82 60 97 98

3594 4313 2491 2468

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 11 of 15

Table 5 Performance for optimizing the combination of four drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f4a(x) (2) A success rateunique comb

21 13 96 98

7989 7117 3939 3274

B success rateiterations

21 12 96 97

9419 10321 5583 4523

(5) A success rateunique comb

21 14 90 95

8163 6536 4731 3983

B success rateiterations

21 13 90 95

8956 10229 6754 5812

(8) A success rateunique comb

24 14 85 95

8587 6816 5057 4336

B success rateiterations

23 13 84 92

9971 10089 7208 6485

f4b(x) (2) A success rateunique comb

62 41 100 100

6345 5231 1380 1031

B success rateiterations

51 26 100 100

9324 9030 2369 1828

(5) A success rateunique comb

75 68 100 100

6102 4680 2311 1509

B success rateiterations

50 25 100 100

8559 9212 4110 2588

(8) A success rateunique comb

86 82 98 100

5258 4301 3142 2159

B success rateiterations

50 24 94 100

8353 9792 6020 3938

Table 6 Performance for optimizing the combination of five drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f5a(x) (2) A success rateunique comb

8 9 100 100

21 44 1393 1225

B success rateiterations

9 11 100 100

21 60 1724 1549

(5) A success rateunique comb

9 11 100 100

39 79 1421 1291

B success rateiterations

9 12 100 100

1083 386 1771 1556

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 12 of 15

Table 6 Performance for optimizing the combination of five drugs in the presence of noise (Continued)

(8) A success rateunique comb

10 13 100 100

71 202 1445 1314

B success rateiterations

9 13 100 100

1914 708 1823 1565

f5b(x) (2) A success rateunique comb

89 55 100 100

9179 10263 4071 3439

B success rateiterations

90 55 100 100

9998 13251 5165 4443

(5) A success rateunique comb

90 59 97 98

9328 10027 5077 4636

B success rateiterations

90 56 99 99

10047 13327 6569 5624

(8) A success rateunique comb

91 59 97 98

9595 9712 5788 5348

B success rateiterations

90 56 99 99

10152 13410 7350 6686

Table 7 Performance for optimizing the combination of six drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f6a(x) (2) A success rateunique comb

91 43 100 100

12800 12141 4768 4326

B success rateiterations

90 43 100 100

13526 16626 5314 5036

(5) A success rateunique comb

90 44 99 99

12623 12472 6210 5983

B success rateiterations

90 45 100 100

13968 16759 7637 7361

(8) A success rateunique comb

90 46 98 98

12047 13024 7232 6988

B success rateiterations

90 46 98 98

14122 16819 8752 8343

f6b(x) (2) A success rateunique comb

91 43 100 100

5099 9710 3414 2377

B success rateiterations

94 44 100 100

12407 16460 4365 2935

(5) A success rateunique comb

90 42 100 100

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 13 of 15

ARU algorithm continued to substantially outperformother stochastic search algorithms [911] demonstratingthat it is better suited for practical drug optimizationapplicationsOne interesting observation is that the performance of

the Gur Game algorithm is typically not very sensitiveto measurement noise In fact in some cases its perfor-mance even improves as the noise level goes up Themain reason for this phenomenon is as follows As dis-cussed earlier the Gur Game algorithm does not adaptto the observed drug response function and for this rea-son its overall performance crucially depends onwhether or not its predetermined FSA matches the drugresponse function at hand As a result if the FSA doesnot match the original drug response function well iro-nically enough the measurement noise may perturb thesearch process in such a way that improves the overallperformance In this sense the fact that the Gur Gamealgorithm is not very sensitive to measurement noisereflects its inaptitude for handling various types of drugresponse functions rather than its robustness to randomfluctuations and noise in the measured drug response

ConclusionsIn this paper we proposed a novel stochastic searchalgorithm called the adaptive reference update (ARU)algorithm which can be effectively used for optimizingthe composition of combinatory drugs The proposedalgorithm intelligently utilizes the drug response valuesobserved in the past to reliably predict how to benefi-cially update the drug concentrations to improve thedrug response As we demonstrated throughout thispaper the proposed algorithm addresses several short-comings of previous drug optimization algorithms[911] thereby improving the overall search perfor-mance Numerical experiments based on various typesof multi-drug response functions show that the ARUalgorithm results in a higher success rate (ie higherprobability of identifying a potent drug combination)while requiring significantly fewer drug response evalua-tions Furthermore the proposed algorithm is robust to

random measurement noise where its search perfor-mance is not substantially affected in the presence ofnoise and degrades gracefully as the noise levelincreases Throughout our experiments we used a = 1for the ARU algorithm as well as the previous searchalgorithm [11] As discussed earlier the parameter acontrols the randomness of the search by determininghow much weight we should give to the drug responsevalues that we observed in the past In general unlessthe observations are very noisy or the underlying drugresponse function is assumed to be extremely nonlinearit would be best to set the parameter to the largestallowed value (ie a = 1) so that we fully utilize thepast observations for making our best informed guessabout the beneficial drug update strategy For compari-son we also repeated our simulations using a = 05 anda = 075 whose results are summarized in Table S1 -Table S7 (see Additional file 1) We can see from theseresults that a = 1 indeed leads to the best performancefor the drug response functions and the noise levels wehave considered in this paper

Additional material

Additional file 1 Performance of the ARU algorithm Furtherperformance evaluation results of the proposed Adaptive ReferenceUpdate (ARU) algorithm

AcknowledgementsBased on ldquoEfficient combinatorial drug optimization through stochasticsearchrdquo by Mansuck Kim and Byung-Jun Yoon which appeared in GenomicSignal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop oncopy 2011 IEEE [19]This work was supported in part by the National Science Foundationthrough NSF Award CCF-1149544This article has been published as part of BMC Genomics Volume 13Supplement 6 2012 Selected articles from the IEEE International Workshopon Genomic Signal Processing and Statistics (GENSIPS) 2011 The fullcontents of the supplement are available online at httpwwwbiomedcentralcombmcgenomicssupplements13S6

Authorsrsquo contributionsConceived and developed the algorithm MK BJY Performed theexperiments MK Analyzed the data and wrote the paper MK BJY

Table 7 Performance for optimizing the combination of six drugs in the presence of noise (Continued)

4738 9707 3499 2798

B success rateiterations

94 44 100 100

12786 17049 4808 3246

(8) A success rateunique comb

89 42 100 100

4544 9699 4574 3532

B success rateiterations

94 44 100 100

13331 17755 5453 3915

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 14 of 15

utilize the past response data to beneficially update thedrug concentrations and ultimately to identify a potentdrug combination while keeping the search still stochas-tic For illustration let us again consider the drugresponse function in Figure 2 where the hypotheticalsearch process proceeds from the lowest drug concen-tration to the highest concentration The black solidarrows below the graph shows the drug update directionthat gets higher probability according to the new algo-rithm described above For example in region-A(cmin lt times lt xref1) the algorithm tends to increase thedrug concentration x further as the response f(x) is lar-ger than f(cmin) of the initial reference concentration(ie cmin) As x continues to increase and passes thefirst local maximum point xref1 the reference is updatedto xref not xref1 In region-B (xref1 lt times lt xref2) the searchalgorithm tends to drive the concentration towards xref1

by decreasing the concentration Suppose the searchcontinues to increase the drug concentration x beyondxref2 the second local maximum point despite the ten-dency of the algorithm to decrease x back to xref1 Afterpassing xref2 the reference is updated to xref not xref2 Inregion-C the search algorithms assigns higher probabil-ity to the update rule that tries to bring the concentra-tion down to xref2 since f(x) lt f(xref2) in the givenregion However once x enters region-D where f(x) gtf(xref2) the algorithm begins to drive the drug concen-tration x further to the right until it passes the thirdlocal maximum point xref3 The reference concentra-tion is updated to xref not xref3 once the search con-tinues to the right and the drug concentration x getslarger than xref3 Since f(xref3) is larger than f(x) inregion-D (xref3 lt times lt cmax) the search algorithm willtend to bring the concentration down to the currentreference concentration namely xref = xref3Choosing a local maximum solution as a reference com-

bination has a number of practical advantages First of allit allows the algorithm to adjust the drug update rulesbased on a long-range trend of the given drug responsefunction which makes the algorithm robust to small varia-tions in the observed response Another advantage ofusing a long-range trend is that the search process willbecome also less sensitive to random fluctuations that mayexist in the observed drug response Considering that thedrug response function f(x) has to be experimentally esti-mated through actual biological experiments where ran-dom factors (eg measurement noise) that affect theestimation results cannot be completely ruled out suchrobustness is critical for the algorithm to be used in practi-cal drug optimization applications It is also beneficial touse the drug combination that corresponds to the mostrecent local maximum response instead of the drug com-bination that has yielded the highest response among allpast combinations as the reference point This prevents

the search process from dwelling too much on past obser-vations while keeping it robust to variations and randomfluctuations

Drug response functionsIn order to evaluate the overall performance of the ARUalgorithm we used the algorithm to search for the opti-mal drug cocktail for several different drug responsefunctionsTwo-dimensional drug response functions For

performance assessment we first used the four two-dimensional drug response functions that are shown inFigure 3 The first drug response function f2a(x1 x2)shown in Figure 3A is the normalized HIV inhibitorresponse obtained from [17] where x1 Icirc 0 001 003009 027 082 247 741 2222 6667(nM) was consid-ered for Maraviroc and x2 Icirc 0 009 027 08 241722 2167 65(nM) for ROAb14 The second drugresponse f2b(x1 x2) shown in Figure 3B is the normal-ized second De Jong function (Rosenbrockrsquos saddle)[18] We considered x1 x2 Icirc c0 c1 c20 where ck =4(k20 - 05) obtained by evenly dividing the range [-22] into 21 distinct values The third drug response func-tion f2c(x1 x2) in Figure 3C is the normalized lung can-cer inhibition response obtained from [1] where x1 Icirc0 1 2 4 6 8 12 16 20 22(μM ) was considered forChlorpromazine and x2 Icirc 0 025 04 06 08 1 15 24 68(μM ) for Pentamidine Finally Figure 3D showsthe fourth response function f2d(x1 x2) the normalizedbacterial (S aureus) inhibition response reported in [6]x1 Icirc 0 008 016 032 063 125 25 5 10 was con-sidered for Trimethoprim and x2 Icirc 0 031 062 12525 5 10 20 40 was considered for SulfamethoxazoleAll four drug response functions were normalized tospan the entire range [0 1] such that the minimumresponse is 0 and the maximum response is 1Multi-dimensional drug response functions To eval-

uate the performance for optimizing multi-drug cocktailswe defined several hypothetical drug response functionswith up to six drugs First we defined two 3-dimensionaldrug functions f3a(x1 x2 x3) and f3b(x1 x2 x3) The firstfunction is defined as

f3a(x1 x2 x3) = x21 middot sin2(x2) middot cos2(x3) (8)

where each of x1 x2 x3 can take one of the 11 discreteconcentrations that evenly divide the range [-25 25]The second function is defined as follows

f3b(x1 x2 x3) = peaks(x1 x2) middot x3 (9)

using the Matlab peaks(x1 x2) function We assumethat each drug can take one of the 11 discrete valuesthat evenly divide [-3 3] Next we defined two 4-dimen-sional drug functions

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 6 of 15

f4a(x1 x2 x3 x4) = x1 middot eminus(x21+x22+x

23+x

24) (10)

and

f4b(x1 x2 x3 x4) = cos2(03x1) middot sin(03x2) middot tan(01x3) middot x4 (11)

We assume that x1 and x2 in the first function f4a(x1x2 x3 x4) can take one of the 11 discrete values thatevenly divide the range [-2 2] while x3 and x4 can takeone of the 11 discrete values that evenly divide therange [-3 3] For the second drug response function f4b(x1 x2 x3 x4) we assume that each drug can take oneof the 11 distinct values that evenly divide [-3 3] Inaddition we also defined the following 5-dimensionaldrug response functions

f5a(x1 x2 x3 x4 x5) = eminusx1 middot cos2(x2) middot x23 middot[eminus(x4+2)

2minus(x5+3)2+ eminus(x4minus2)2minus(x5minus3)2

](12)

and

f5b(x1 x2 x3 x4 x5) =12peaks(x1 x2) middot cos(05x3) middot sin(05x4) middot x25 (13)

For the first function f5a(x1 x2 x3 x4 x5) x1 and x2 areallowed to take any value from the set of values obtainedby evenly dividing the range [-2 2] into 11 discrete con-centrations The remaining drug concentrations (x3 x4and x5) can take any of the 11 concentrations that evenlydivide [-45 45] For the second drug response functionf5b(x1 x2 x3 x4 x5) we assume that each drug can takeone of the 11 discrete values that evenly divide [-3 3]Finally we also defined two 6-dimensional drug responsefunctions

f6a(x1 x2 x3 x4 x5 x6) = eminus075(x1) middot (sin2(x2) + cos(x3)

) middot eminus075(x24+x25) middot x6 (14)

and

f6b(x1 x2 x3 x4 x5 x6) = eminus01(x21minusx22)minus01(x23+x24) middot cos2(02x35) middot sin(02x36) (15)

where every drug concentration can take its valuefrom one of the 11 discrete concentrations that evenlydivide the range [-25 25]

Figure 3 Two-dimensional drug response functions (A) Inhibition of HIV (B) Second De Jong function (Rosenbrockrsquos saddle) (C) Inhibition ofA549 lung carcinoma cell proliferation (D) Inhibition of bacteria (S aureus) proliferation

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 7 of 15

ResultsOptimizing the combination of two drugsWe first evaluated the overall performance of theARU stochastic search algorithm based on four two-dimensional drug response functions (see Methods) (i)HIV inhibitor response f2a(x1 x2) (ii) second De Jongfunction (Rosenbrockrsquos saddle) f2b(x1 x2) (iii) normal-ized lung cancer inhibition response f2c(x1 x2) and (iv)bacterial (S aureus) inhibition response f2d(x1 x2)These functions are shown in Figure 3 For each drugresponse function we searched for the optimal drugresponse using the proposed algorithm starting fromrandomly selected drug concentrations x1 and x2 Theparameter a that controls the randomness of the searchwas set to a = 1 which implies that the algorithm adap-tively determines the search direction (ie how toupdate the current drug concentration) by fully utilizingthe past observations Note that setting the parameter toa = 0 in (7) would make the search completely randomand independent of the past observations at each stepthe concentration of a given drug will be randomlyincreased or decreased with equal probability regardlessof the update rules given in (5) In each search experi-ment the iterations were repeated up to two times thetotal number of possible drug combinations Thisexperiment was repeated 5000 times to obtain reliableresults For comparison we performed similar experi-ments using the Gur Game algorithm [9] and the sto-chastic search algorithm proposed in [11] with a = 1We computed the following two performance metricsthe success rate and the average number of unique drugcombinations that need to be tested The first metric isdefined as the relative proportion of experiments inwhich the algorithm was able to identify a potent drugcombination whose response is within 5 of the maxi-mum response ie f(x1 x2) ge 095 The second metricis defined as the average number of unique drug combi-nations that have to be tested until a potent drug com-bination is identified in case the search is successfulThese performance assessment results are summarizedin Table 1 Note that the Gur Game algorithm has beentested using both the simultaneous update strategy as

well as the sequential update strategy Unlike the ARUalgorithm and the search algorithm proposed in [11]which update one drug at a time (ie ldquosequentialrdquo drugupdate) the original Gur Game algorithm adopted in[9] updates all drugs simultaneously However it is alsopossible to use the sequential update strategy with theGur Game algorithm and we have evaluated both stra-tegies for comparison Table 1 shows that the proposedARU algorithm outperforms the existing algorithms interms of success rate Furthermore when the successrates are comparable the ARU algorithm can in generalidentify an effective drug combination more efficientlyas reflected in the smaller number of unique drug com-binations that need to be tested We can get a morecomplete picture of the efficiency of the ARU algorithmfrom Figure S1 and Figure S2 which respectively showthe distribution of the number of unique drug combina-tions that need to be tested and the distribution of thenumber of iterations that are needed to identify a potentdrug combination (see Additional file 1)

Optimizing multi-drug cocktailsNext we tested the performance of the ARU algorithmfor optimizing multi-drug cocktails that consist of threeto six drugs For this purpose we used the eight hypothe-tical drug response functions that were defined before(see Methods) As in our previous experiments for eachdrug response function we used the proposed ARU algo-rithm (with a = 1) to search for a potent drug combina-tion whose response is within 5 of the maximumresponse (ie f(x) ge 095) In each search experiment westarted from an randomly selected initial concentrationsand continued the search up to 1000 steps for 3 drugs2000 steps for 4 drugs 3000 steps for 5 drugs and 4000steps for 6 drugs This experiment was repeated 5000times to obtain reliable performance assessment resultsThe simulation results are summarized in Table 2 Asshown in this table the proposed algorithm boasts a sig-nificantly higher success rate compared to the Gur Gamealgorithm [9] It also results in either comparable orslightly improved success rate compared to the previousdrug optimization algorithm (a = 1) [11] However we

Table 1 Performance for optimizing the combination of two drugs

Gur Game(simultaneous)

Gur Game (sequential) Previous searchalgorithm [11](a = 1)

ARU algorithm(proposed) (a = 1)

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

f2a(x) HIV INHIBITION (M = 80) 97 132 95 172 100 134 100 121

f2b(x) DEJONG (2ND) (M = 441) 9 36 10 45 99 562 99 462

f2c(x) CANCERINHIBITION

(M = 100) 58 113 53 130 98 132 98 124

f2d(x) BACTERIAINHIBITION

(M = 81) 96 59 91 68 100 48 100 45

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 8 of 15

can see that the ARU algorithm clearly outperforms theprevious search algorithm in terms of efficiency asreflected in the significantly smaller number of uniquedrug combinations that need to be tested until an effec-tive combination is identified Figures S3 and S5 showthe distribution of the number of unique drug combina-tions that need to be tested to identify an effective drugcocktail (see Additional file 1) Similarly Figures S4 andS6 show the distribution of the number of search itera-tions that are needed by each algorithm

Drug optimization in the presence of measurement noiseIn order to use a drug optimization algorithm in practi-cal applications the algorithm has to be robust to ran-dom fluctuations in the estimated drug response Toevaluate the robustness of the proposed ARU algorithmwe evaluated its search performance in the presence ofmeasurement noise and compared it with other existingstochastic search algorithms In these experiments weconsidered two different types of search strategies Inthe first search strategy (referred as type-A) when thesearch algorithm happens to revisit a drug combinationthat was previously tested it does not re-evaluate thedrug response and simply uses the previously estimatedvalue On the other hand according to the second strat-egy (referred as type-B) the search algorithm always re-evaluates the drug response even if it revisits a pre-viously evaluated drug combination since the measuredresponse may be different every time due to the randommeasurement noise The first strategy may be usefulwhen the noise level is relatively low in which case thisstrategy may be able to reduce the total number of drugresponse evaluations thereby reducing the overallexperimental cost for identifying a potent drug combi-nation However when the noise level is high thesearch performance may be degraded as the search algo-rithm clings to the past (noisy) response once it hasbeen measured In contrast the second search strategygenerally requires a relatively larger number of drugresponse evaluations but it tends to be more robust to

random fluctuations and noise in the measured drugresponse functionIn order to evaluate the performance of the different

search algorithms in the presence of noise we per-formed similar search experiments as before at three dif-ferent levels of additive noise 2 5 and 8 Moreprecisely we assume that

fobs(x) = ftrue(x) + η

where fobs(x) is the observed drug response ftrue(x) is thetrue response and h is an independent random noise thatis uniformly distributed over (-u u) where u Icirc 002 005008 For each drug response function and a given noiselevel u we tested the performance of both search strate-gies For type-A search we evaluated the success rate andthe average number of unique drug combinations thathave to be tested until a potent drug combination is identi-fied For type-B search we evaluated the success rate andthe average number of iterations instead of the number ofunique drug combinations until an effective combinationis identified This is because in a type-B search the searchalgorithm re-evaluates the drug response even if it revisitsthe same drug combination that was previously tested Thesimulation results are shown in Table 3 4 5 6 7 for drugresponse functions with two to six drugs The parameter awas set to a = 1 for the ARU algorithm as well as the pre-vious search algorithm proposed in [11]As we can see in these Tables measurement noise cer-

tainly affects the overall performance of the ARU algo-rithm where a higher noise tends to reduce the successrate and increase the number of iterations as well as thatof the unique drug combinations to be tested For manydrug response functions considered in our simulationsthe performance degradation is typically not too signifi-cant for the proposed algorithm showing that the ARUalgorithm is relatively robust to measurement noiseHowever we can also observe that the extent of perfor-mance degradation will critically depend on the land-scape of the underlying drug response In most cases the

Table 2 Performance for optimizing multi-drug cocktails

Gur Game (simultaneous) Gur Game (sequential) Previous search algorithm [11](a = 1)

ARU algorithm (proposed)(a = 1)

success rate unique comb success rate unique comb success rate unique comb success rate unique comb

f3a(x) (M = 113) 1 43 2 55 100 1053 100 740

f3b(x) (M = 113) 83 2294 58 2048 100 885 100 794

f4a(x) (M = 114) 20 8239 11 6666 100 1779 100 1368

f4b(x) (M = 114) 52 7067 24 5209 100 1179 100 916

f5a(x) (M = 115) 8 21 2 48 100 1381 100 806

f5b(x) (M = 115) 89 10134 54 9762 100 2529 100 2168

f6a(x) (M = 116) 90 12691 44 12608 100 1919 100 1781

f6b(x) (M = 116) 90 4467 40 10332 100 2381 100 1901

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 9 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f2a(x) (2) A success rateunique comb

97 96 100 100

125 170 133 116

B success rateiterations

97 95 100 100

378 452 258 200

(5) A success rateunique comb

97 96 100 100

126 171 133 118

B success rateiterations

97 95 100 100

380 454 266 202

(8) A success rateunique comb

97 96 100 100

126 170 133 120

B success rateiterations

97 95 100 100

382 454 268 204

f2b(x) (2) A success rateunique comb

10 10 99 99

39 42 570 451

B success rateiterations

10 10 99 99

40 44 1485 1200

(5) A success rateunique comb

9 9 98 98

41 45 629 522

B success rateiterations

9 9 98 99

43 47 1721 1438

(8) A success rateunique comb

8 9 97 98

41 47 662 556

B success rateiterations

9 9 97 98

44 49 1981 1673

f2c(x) (2) A success rateunique comb

61 54 98 98

109 127 131 125

B success rateiterations

60 54 98 98

331 360 359 352

(5) A success rateunique comb

71 66 98 98

114 132 129 124

B success rateiterations

60 54 98 98

332 354 367 360

(8) A success rateunique comb

88 83 98 98

117 134 126 120

B success rateiterations

6 54 98 98

334 343 374 370

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 10 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise (Continued)

f2d(x) (2) A success rateunique comb

100 100 100 100

49 60 46 41

B success rateiterations

96 9 100 100

183 197 82 77

(5) A success rateunique comb

100 100 100 100

49 57 43 41

B success rateiterations

96 91 100 100

184 196 81 75

(8) A success rateunique comb

100 100 100 100

48 56 44 42

B success rateiterations

96 91 100 100

186 196 81 71

Table 4 Performance for optimizing the combination of three drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f3a(x) (2) A success rateunique comb

1 3 99 100

24 73 1106 773

B success rateiterations

1 3 99 100

28 109 2011 1396

(5) A success rateunique comb

1 3 99 100

24 74 1117 784

B success rateiterations

1 3 99 100

25 101 2016 1440

(8) A success rateunique comb

1 3 99 100

25 77 1133 805

B success rateiterations

1 3 99 100

23 94 2109 1517

f3b(x) (2) A success rateunique comb

86 69 99 99

2243 2016 1108 933

B success rateiterations

83 59 99 99

3671 4191 2115 2053

(5) A success rateunique comb

89 72 99 99

2223 2016 1166 1062

B success rateiterations

83 59 98 98

3593 4399 2255 2229

(8) A success rateunique comb

90 74 97 98

2259 2009 1264 1143

B success rateiterations

82 60 97 98

3594 4313 2491 2468

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 11 of 15

Table 5 Performance for optimizing the combination of four drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f4a(x) (2) A success rateunique comb

21 13 96 98

7989 7117 3939 3274

B success rateiterations

21 12 96 97

9419 10321 5583 4523

(5) A success rateunique comb

21 14 90 95

8163 6536 4731 3983

B success rateiterations

21 13 90 95

8956 10229 6754 5812

(8) A success rateunique comb

24 14 85 95

8587 6816 5057 4336

B success rateiterations

23 13 84 92

9971 10089 7208 6485

f4b(x) (2) A success rateunique comb

62 41 100 100

6345 5231 1380 1031

B success rateiterations

51 26 100 100

9324 9030 2369 1828

(5) A success rateunique comb

75 68 100 100

6102 4680 2311 1509

B success rateiterations

50 25 100 100

8559 9212 4110 2588

(8) A success rateunique comb

86 82 98 100

5258 4301 3142 2159

B success rateiterations

50 24 94 100

8353 9792 6020 3938

Table 6 Performance for optimizing the combination of five drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f5a(x) (2) A success rateunique comb

8 9 100 100

21 44 1393 1225

B success rateiterations

9 11 100 100

21 60 1724 1549

(5) A success rateunique comb

9 11 100 100

39 79 1421 1291

B success rateiterations

9 12 100 100

1083 386 1771 1556

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 12 of 15

Table 6 Performance for optimizing the combination of five drugs in the presence of noise (Continued)

(8) A success rateunique comb

10 13 100 100

71 202 1445 1314

B success rateiterations

9 13 100 100

1914 708 1823 1565

f5b(x) (2) A success rateunique comb

89 55 100 100

9179 10263 4071 3439

B success rateiterations

90 55 100 100

9998 13251 5165 4443

(5) A success rateunique comb

90 59 97 98

9328 10027 5077 4636

B success rateiterations

90 56 99 99

10047 13327 6569 5624

(8) A success rateunique comb

91 59 97 98

9595 9712 5788 5348

B success rateiterations

90 56 99 99

10152 13410 7350 6686

Table 7 Performance for optimizing the combination of six drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f6a(x) (2) A success rateunique comb

91 43 100 100

12800 12141 4768 4326

B success rateiterations

90 43 100 100

13526 16626 5314 5036

(5) A success rateunique comb

90 44 99 99

12623 12472 6210 5983

B success rateiterations

90 45 100 100

13968 16759 7637 7361

(8) A success rateunique comb

90 46 98 98

12047 13024 7232 6988

B success rateiterations

90 46 98 98

14122 16819 8752 8343

f6b(x) (2) A success rateunique comb

91 43 100 100

5099 9710 3414 2377

B success rateiterations

94 44 100 100

12407 16460 4365 2935

(5) A success rateunique comb

90 42 100 100

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 13 of 15

ARU algorithm continued to substantially outperformother stochastic search algorithms [911] demonstratingthat it is better suited for practical drug optimizationapplicationsOne interesting observation is that the performance of

the Gur Game algorithm is typically not very sensitiveto measurement noise In fact in some cases its perfor-mance even improves as the noise level goes up Themain reason for this phenomenon is as follows As dis-cussed earlier the Gur Game algorithm does not adaptto the observed drug response function and for this rea-son its overall performance crucially depends onwhether or not its predetermined FSA matches the drugresponse function at hand As a result if the FSA doesnot match the original drug response function well iro-nically enough the measurement noise may perturb thesearch process in such a way that improves the overallperformance In this sense the fact that the Gur Gamealgorithm is not very sensitive to measurement noisereflects its inaptitude for handling various types of drugresponse functions rather than its robustness to randomfluctuations and noise in the measured drug response

ConclusionsIn this paper we proposed a novel stochastic searchalgorithm called the adaptive reference update (ARU)algorithm which can be effectively used for optimizingthe composition of combinatory drugs The proposedalgorithm intelligently utilizes the drug response valuesobserved in the past to reliably predict how to benefi-cially update the drug concentrations to improve thedrug response As we demonstrated throughout thispaper the proposed algorithm addresses several short-comings of previous drug optimization algorithms[911] thereby improving the overall search perfor-mance Numerical experiments based on various typesof multi-drug response functions show that the ARUalgorithm results in a higher success rate (ie higherprobability of identifying a potent drug combination)while requiring significantly fewer drug response evalua-tions Furthermore the proposed algorithm is robust to

random measurement noise where its search perfor-mance is not substantially affected in the presence ofnoise and degrades gracefully as the noise levelincreases Throughout our experiments we used a = 1for the ARU algorithm as well as the previous searchalgorithm [11] As discussed earlier the parameter acontrols the randomness of the search by determininghow much weight we should give to the drug responsevalues that we observed in the past In general unlessthe observations are very noisy or the underlying drugresponse function is assumed to be extremely nonlinearit would be best to set the parameter to the largestallowed value (ie a = 1) so that we fully utilize thepast observations for making our best informed guessabout the beneficial drug update strategy For compari-son we also repeated our simulations using a = 05 anda = 075 whose results are summarized in Table S1 -Table S7 (see Additional file 1) We can see from theseresults that a = 1 indeed leads to the best performancefor the drug response functions and the noise levels wehave considered in this paper

Additional material

Additional file 1 Performance of the ARU algorithm Furtherperformance evaluation results of the proposed Adaptive ReferenceUpdate (ARU) algorithm

AcknowledgementsBased on ldquoEfficient combinatorial drug optimization through stochasticsearchrdquo by Mansuck Kim and Byung-Jun Yoon which appeared in GenomicSignal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop oncopy 2011 IEEE [19]This work was supported in part by the National Science Foundationthrough NSF Award CCF-1149544This article has been published as part of BMC Genomics Volume 13Supplement 6 2012 Selected articles from the IEEE International Workshopon Genomic Signal Processing and Statistics (GENSIPS) 2011 The fullcontents of the supplement are available online at httpwwwbiomedcentralcombmcgenomicssupplements13S6

Authorsrsquo contributionsConceived and developed the algorithm MK BJY Performed theexperiments MK Analyzed the data and wrote the paper MK BJY

Table 7 Performance for optimizing the combination of six drugs in the presence of noise (Continued)

4738 9707 3499 2798

B success rateiterations

94 44 100 100

12786 17049 4808 3246

(8) A success rateunique comb

89 42 100 100

4544 9699 4574 3532

B success rateiterations

94 44 100 100

13331 17755 5453 3915

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 14 of 15

f4a(x1 x2 x3 x4) = x1 middot eminus(x21+x22+x

23+x

24) (10)

and

f4b(x1 x2 x3 x4) = cos2(03x1) middot sin(03x2) middot tan(01x3) middot x4 (11)

We assume that x1 and x2 in the first function f4a(x1x2 x3 x4) can take one of the 11 discrete values thatevenly divide the range [-2 2] while x3 and x4 can takeone of the 11 discrete values that evenly divide therange [-3 3] For the second drug response function f4b(x1 x2 x3 x4) we assume that each drug can take oneof the 11 distinct values that evenly divide [-3 3] Inaddition we also defined the following 5-dimensionaldrug response functions

f5a(x1 x2 x3 x4 x5) = eminusx1 middot cos2(x2) middot x23 middot[eminus(x4+2)

2minus(x5+3)2+ eminus(x4minus2)2minus(x5minus3)2

](12)

and

f5b(x1 x2 x3 x4 x5) =12peaks(x1 x2) middot cos(05x3) middot sin(05x4) middot x25 (13)

For the first function f5a(x1 x2 x3 x4 x5) x1 and x2 areallowed to take any value from the set of values obtainedby evenly dividing the range [-2 2] into 11 discrete con-centrations The remaining drug concentrations (x3 x4and x5) can take any of the 11 concentrations that evenlydivide [-45 45] For the second drug response functionf5b(x1 x2 x3 x4 x5) we assume that each drug can takeone of the 11 discrete values that evenly divide [-3 3]Finally we also defined two 6-dimensional drug responsefunctions

f6a(x1 x2 x3 x4 x5 x6) = eminus075(x1) middot (sin2(x2) + cos(x3)

) middot eminus075(x24+x25) middot x6 (14)

and

f6b(x1 x2 x3 x4 x5 x6) = eminus01(x21minusx22)minus01(x23+x24) middot cos2(02x35) middot sin(02x36) (15)

where every drug concentration can take its valuefrom one of the 11 discrete concentrations that evenlydivide the range [-25 25]

Figure 3 Two-dimensional drug response functions (A) Inhibition of HIV (B) Second De Jong function (Rosenbrockrsquos saddle) (C) Inhibition ofA549 lung carcinoma cell proliferation (D) Inhibition of bacteria (S aureus) proliferation

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 7 of 15

ResultsOptimizing the combination of two drugsWe first evaluated the overall performance of theARU stochastic search algorithm based on four two-dimensional drug response functions (see Methods) (i)HIV inhibitor response f2a(x1 x2) (ii) second De Jongfunction (Rosenbrockrsquos saddle) f2b(x1 x2) (iii) normal-ized lung cancer inhibition response f2c(x1 x2) and (iv)bacterial (S aureus) inhibition response f2d(x1 x2)These functions are shown in Figure 3 For each drugresponse function we searched for the optimal drugresponse using the proposed algorithm starting fromrandomly selected drug concentrations x1 and x2 Theparameter a that controls the randomness of the searchwas set to a = 1 which implies that the algorithm adap-tively determines the search direction (ie how toupdate the current drug concentration) by fully utilizingthe past observations Note that setting the parameter toa = 0 in (7) would make the search completely randomand independent of the past observations at each stepthe concentration of a given drug will be randomlyincreased or decreased with equal probability regardlessof the update rules given in (5) In each search experi-ment the iterations were repeated up to two times thetotal number of possible drug combinations Thisexperiment was repeated 5000 times to obtain reliableresults For comparison we performed similar experi-ments using the Gur Game algorithm [9] and the sto-chastic search algorithm proposed in [11] with a = 1We computed the following two performance metricsthe success rate and the average number of unique drugcombinations that need to be tested The first metric isdefined as the relative proportion of experiments inwhich the algorithm was able to identify a potent drugcombination whose response is within 5 of the maxi-mum response ie f(x1 x2) ge 095 The second metricis defined as the average number of unique drug combi-nations that have to be tested until a potent drug com-bination is identified in case the search is successfulThese performance assessment results are summarizedin Table 1 Note that the Gur Game algorithm has beentested using both the simultaneous update strategy as

well as the sequential update strategy Unlike the ARUalgorithm and the search algorithm proposed in [11]which update one drug at a time (ie ldquosequentialrdquo drugupdate) the original Gur Game algorithm adopted in[9] updates all drugs simultaneously However it is alsopossible to use the sequential update strategy with theGur Game algorithm and we have evaluated both stra-tegies for comparison Table 1 shows that the proposedARU algorithm outperforms the existing algorithms interms of success rate Furthermore when the successrates are comparable the ARU algorithm can in generalidentify an effective drug combination more efficientlyas reflected in the smaller number of unique drug com-binations that need to be tested We can get a morecomplete picture of the efficiency of the ARU algorithmfrom Figure S1 and Figure S2 which respectively showthe distribution of the number of unique drug combina-tions that need to be tested and the distribution of thenumber of iterations that are needed to identify a potentdrug combination (see Additional file 1)

Optimizing multi-drug cocktailsNext we tested the performance of the ARU algorithmfor optimizing multi-drug cocktails that consist of threeto six drugs For this purpose we used the eight hypothe-tical drug response functions that were defined before(see Methods) As in our previous experiments for eachdrug response function we used the proposed ARU algo-rithm (with a = 1) to search for a potent drug combina-tion whose response is within 5 of the maximumresponse (ie f(x) ge 095) In each search experiment westarted from an randomly selected initial concentrationsand continued the search up to 1000 steps for 3 drugs2000 steps for 4 drugs 3000 steps for 5 drugs and 4000steps for 6 drugs This experiment was repeated 5000times to obtain reliable performance assessment resultsThe simulation results are summarized in Table 2 Asshown in this table the proposed algorithm boasts a sig-nificantly higher success rate compared to the Gur Gamealgorithm [9] It also results in either comparable orslightly improved success rate compared to the previousdrug optimization algorithm (a = 1) [11] However we

Table 1 Performance for optimizing the combination of two drugs

Gur Game(simultaneous)

Gur Game (sequential) Previous searchalgorithm [11](a = 1)

ARU algorithm(proposed) (a = 1)

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

f2a(x) HIV INHIBITION (M = 80) 97 132 95 172 100 134 100 121

f2b(x) DEJONG (2ND) (M = 441) 9 36 10 45 99 562 99 462

f2c(x) CANCERINHIBITION

(M = 100) 58 113 53 130 98 132 98 124

f2d(x) BACTERIAINHIBITION

(M = 81) 96 59 91 68 100 48 100 45

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 8 of 15

can see that the ARU algorithm clearly outperforms theprevious search algorithm in terms of efficiency asreflected in the significantly smaller number of uniquedrug combinations that need to be tested until an effec-tive combination is identified Figures S3 and S5 showthe distribution of the number of unique drug combina-tions that need to be tested to identify an effective drugcocktail (see Additional file 1) Similarly Figures S4 andS6 show the distribution of the number of search itera-tions that are needed by each algorithm

Drug optimization in the presence of measurement noiseIn order to use a drug optimization algorithm in practi-cal applications the algorithm has to be robust to ran-dom fluctuations in the estimated drug response Toevaluate the robustness of the proposed ARU algorithmwe evaluated its search performance in the presence ofmeasurement noise and compared it with other existingstochastic search algorithms In these experiments weconsidered two different types of search strategies Inthe first search strategy (referred as type-A) when thesearch algorithm happens to revisit a drug combinationthat was previously tested it does not re-evaluate thedrug response and simply uses the previously estimatedvalue On the other hand according to the second strat-egy (referred as type-B) the search algorithm always re-evaluates the drug response even if it revisits a pre-viously evaluated drug combination since the measuredresponse may be different every time due to the randommeasurement noise The first strategy may be usefulwhen the noise level is relatively low in which case thisstrategy may be able to reduce the total number of drugresponse evaluations thereby reducing the overallexperimental cost for identifying a potent drug combi-nation However when the noise level is high thesearch performance may be degraded as the search algo-rithm clings to the past (noisy) response once it hasbeen measured In contrast the second search strategygenerally requires a relatively larger number of drugresponse evaluations but it tends to be more robust to

random fluctuations and noise in the measured drugresponse functionIn order to evaluate the performance of the different

search algorithms in the presence of noise we per-formed similar search experiments as before at three dif-ferent levels of additive noise 2 5 and 8 Moreprecisely we assume that

fobs(x) = ftrue(x) + η

where fobs(x) is the observed drug response ftrue(x) is thetrue response and h is an independent random noise thatis uniformly distributed over (-u u) where u Icirc 002 005008 For each drug response function and a given noiselevel u we tested the performance of both search strate-gies For type-A search we evaluated the success rate andthe average number of unique drug combinations thathave to be tested until a potent drug combination is identi-fied For type-B search we evaluated the success rate andthe average number of iterations instead of the number ofunique drug combinations until an effective combinationis identified This is because in a type-B search the searchalgorithm re-evaluates the drug response even if it revisitsthe same drug combination that was previously tested Thesimulation results are shown in Table 3 4 5 6 7 for drugresponse functions with two to six drugs The parameter awas set to a = 1 for the ARU algorithm as well as the pre-vious search algorithm proposed in [11]As we can see in these Tables measurement noise cer-

tainly affects the overall performance of the ARU algo-rithm where a higher noise tends to reduce the successrate and increase the number of iterations as well as thatof the unique drug combinations to be tested For manydrug response functions considered in our simulationsthe performance degradation is typically not too signifi-cant for the proposed algorithm showing that the ARUalgorithm is relatively robust to measurement noiseHowever we can also observe that the extent of perfor-mance degradation will critically depend on the land-scape of the underlying drug response In most cases the

Table 2 Performance for optimizing multi-drug cocktails

Gur Game (simultaneous) Gur Game (sequential) Previous search algorithm [11](a = 1)

ARU algorithm (proposed)(a = 1)

success rate unique comb success rate unique comb success rate unique comb success rate unique comb

f3a(x) (M = 113) 1 43 2 55 100 1053 100 740

f3b(x) (M = 113) 83 2294 58 2048 100 885 100 794

f4a(x) (M = 114) 20 8239 11 6666 100 1779 100 1368

f4b(x) (M = 114) 52 7067 24 5209 100 1179 100 916

f5a(x) (M = 115) 8 21 2 48 100 1381 100 806

f5b(x) (M = 115) 89 10134 54 9762 100 2529 100 2168

f6a(x) (M = 116) 90 12691 44 12608 100 1919 100 1781

f6b(x) (M = 116) 90 4467 40 10332 100 2381 100 1901

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 9 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f2a(x) (2) A success rateunique comb

97 96 100 100

125 170 133 116

B success rateiterations

97 95 100 100

378 452 258 200

(5) A success rateunique comb

97 96 100 100

126 171 133 118

B success rateiterations

97 95 100 100

380 454 266 202

(8) A success rateunique comb

97 96 100 100

126 170 133 120

B success rateiterations

97 95 100 100

382 454 268 204

f2b(x) (2) A success rateunique comb

10 10 99 99

39 42 570 451

B success rateiterations

10 10 99 99

40 44 1485 1200

(5) A success rateunique comb

9 9 98 98

41 45 629 522

B success rateiterations

9 9 98 99

43 47 1721 1438

(8) A success rateunique comb

8 9 97 98

41 47 662 556

B success rateiterations

9 9 97 98

44 49 1981 1673

f2c(x) (2) A success rateunique comb

61 54 98 98

109 127 131 125

B success rateiterations

60 54 98 98

331 360 359 352

(5) A success rateunique comb

71 66 98 98

114 132 129 124

B success rateiterations

60 54 98 98

332 354 367 360

(8) A success rateunique comb

88 83 98 98

117 134 126 120

B success rateiterations

6 54 98 98

334 343 374 370

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 10 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise (Continued)

f2d(x) (2) A success rateunique comb

100 100 100 100

49 60 46 41

B success rateiterations

96 9 100 100

183 197 82 77

(5) A success rateunique comb

100 100 100 100

49 57 43 41

B success rateiterations

96 91 100 100

184 196 81 75

(8) A success rateunique comb

100 100 100 100

48 56 44 42

B success rateiterations

96 91 100 100

186 196 81 71

Table 4 Performance for optimizing the combination of three drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f3a(x) (2) A success rateunique comb

1 3 99 100

24 73 1106 773

B success rateiterations

1 3 99 100

28 109 2011 1396

(5) A success rateunique comb

1 3 99 100

24 74 1117 784

B success rateiterations

1 3 99 100

25 101 2016 1440

(8) A success rateunique comb

1 3 99 100

25 77 1133 805

B success rateiterations

1 3 99 100

23 94 2109 1517

f3b(x) (2) A success rateunique comb

86 69 99 99

2243 2016 1108 933

B success rateiterations

83 59 99 99

3671 4191 2115 2053

(5) A success rateunique comb

89 72 99 99

2223 2016 1166 1062

B success rateiterations

83 59 98 98

3593 4399 2255 2229

(8) A success rateunique comb

90 74 97 98

2259 2009 1264 1143

B success rateiterations

82 60 97 98

3594 4313 2491 2468

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 11 of 15

Table 5 Performance for optimizing the combination of four drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f4a(x) (2) A success rateunique comb

21 13 96 98

7989 7117 3939 3274

B success rateiterations

21 12 96 97

9419 10321 5583 4523

(5) A success rateunique comb

21 14 90 95

8163 6536 4731 3983

B success rateiterations

21 13 90 95

8956 10229 6754 5812

(8) A success rateunique comb

24 14 85 95

8587 6816 5057 4336

B success rateiterations

23 13 84 92

9971 10089 7208 6485

f4b(x) (2) A success rateunique comb

62 41 100 100

6345 5231 1380 1031

B success rateiterations

51 26 100 100

9324 9030 2369 1828

(5) A success rateunique comb

75 68 100 100

6102 4680 2311 1509

B success rateiterations

50 25 100 100

8559 9212 4110 2588

(8) A success rateunique comb

86 82 98 100

5258 4301 3142 2159

B success rateiterations

50 24 94 100

8353 9792 6020 3938

Table 6 Performance for optimizing the combination of five drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f5a(x) (2) A success rateunique comb

8 9 100 100

21 44 1393 1225

B success rateiterations

9 11 100 100

21 60 1724 1549

(5) A success rateunique comb

9 11 100 100

39 79 1421 1291

B success rateiterations

9 12 100 100

1083 386 1771 1556

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 12 of 15

Table 6 Performance for optimizing the combination of five drugs in the presence of noise (Continued)

(8) A success rateunique comb

10 13 100 100

71 202 1445 1314

B success rateiterations

9 13 100 100

1914 708 1823 1565

f5b(x) (2) A success rateunique comb

89 55 100 100

9179 10263 4071 3439

B success rateiterations

90 55 100 100

9998 13251 5165 4443

(5) A success rateunique comb

90 59 97 98

9328 10027 5077 4636

B success rateiterations

90 56 99 99

10047 13327 6569 5624

(8) A success rateunique comb

91 59 97 98

9595 9712 5788 5348

B success rateiterations

90 56 99 99

10152 13410 7350 6686

Table 7 Performance for optimizing the combination of six drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f6a(x) (2) A success rateunique comb

91 43 100 100

12800 12141 4768 4326

B success rateiterations

90 43 100 100

13526 16626 5314 5036

(5) A success rateunique comb

90 44 99 99

12623 12472 6210 5983

B success rateiterations

90 45 100 100

13968 16759 7637 7361

(8) A success rateunique comb

90 46 98 98

12047 13024 7232 6988

B success rateiterations

90 46 98 98

14122 16819 8752 8343

f6b(x) (2) A success rateunique comb

91 43 100 100

5099 9710 3414 2377

B success rateiterations

94 44 100 100

12407 16460 4365 2935

(5) A success rateunique comb

90 42 100 100

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 13 of 15

ARU algorithm continued to substantially outperformother stochastic search algorithms [911] demonstratingthat it is better suited for practical drug optimizationapplicationsOne interesting observation is that the performance of

the Gur Game algorithm is typically not very sensitiveto measurement noise In fact in some cases its perfor-mance even improves as the noise level goes up Themain reason for this phenomenon is as follows As dis-cussed earlier the Gur Game algorithm does not adaptto the observed drug response function and for this rea-son its overall performance crucially depends onwhether or not its predetermined FSA matches the drugresponse function at hand As a result if the FSA doesnot match the original drug response function well iro-nically enough the measurement noise may perturb thesearch process in such a way that improves the overallperformance In this sense the fact that the Gur Gamealgorithm is not very sensitive to measurement noisereflects its inaptitude for handling various types of drugresponse functions rather than its robustness to randomfluctuations and noise in the measured drug response

ConclusionsIn this paper we proposed a novel stochastic searchalgorithm called the adaptive reference update (ARU)algorithm which can be effectively used for optimizingthe composition of combinatory drugs The proposedalgorithm intelligently utilizes the drug response valuesobserved in the past to reliably predict how to benefi-cially update the drug concentrations to improve thedrug response As we demonstrated throughout thispaper the proposed algorithm addresses several short-comings of previous drug optimization algorithms[911] thereby improving the overall search perfor-mance Numerical experiments based on various typesof multi-drug response functions show that the ARUalgorithm results in a higher success rate (ie higherprobability of identifying a potent drug combination)while requiring significantly fewer drug response evalua-tions Furthermore the proposed algorithm is robust to

random measurement noise where its search perfor-mance is not substantially affected in the presence ofnoise and degrades gracefully as the noise levelincreases Throughout our experiments we used a = 1for the ARU algorithm as well as the previous searchalgorithm [11] As discussed earlier the parameter acontrols the randomness of the search by determininghow much weight we should give to the drug responsevalues that we observed in the past In general unlessthe observations are very noisy or the underlying drugresponse function is assumed to be extremely nonlinearit would be best to set the parameter to the largestallowed value (ie a = 1) so that we fully utilize thepast observations for making our best informed guessabout the beneficial drug update strategy For compari-son we also repeated our simulations using a = 05 anda = 075 whose results are summarized in Table S1 -Table S7 (see Additional file 1) We can see from theseresults that a = 1 indeed leads to the best performancefor the drug response functions and the noise levels wehave considered in this paper

Additional material

Additional file 1 Performance of the ARU algorithm Furtherperformance evaluation results of the proposed Adaptive ReferenceUpdate (ARU) algorithm

AcknowledgementsBased on ldquoEfficient combinatorial drug optimization through stochasticsearchrdquo by Mansuck Kim and Byung-Jun Yoon which appeared in GenomicSignal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop oncopy 2011 IEEE [19]This work was supported in part by the National Science Foundationthrough NSF Award CCF-1149544This article has been published as part of BMC Genomics Volume 13Supplement 6 2012 Selected articles from the IEEE International Workshopon Genomic Signal Processing and Statistics (GENSIPS) 2011 The fullcontents of the supplement are available online at httpwwwbiomedcentralcombmcgenomicssupplements13S6

Authorsrsquo contributionsConceived and developed the algorithm MK BJY Performed theexperiments MK Analyzed the data and wrote the paper MK BJY

Table 7 Performance for optimizing the combination of six drugs in the presence of noise (Continued)

4738 9707 3499 2798

B success rateiterations

94 44 100 100

12786 17049 4808 3246

(8) A success rateunique comb

89 42 100 100

4544 9699 4574 3532

B success rateiterations

94 44 100 100

13331 17755 5453 3915

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 14 of 15

ResultsOptimizing the combination of two drugsWe first evaluated the overall performance of theARU stochastic search algorithm based on four two-dimensional drug response functions (see Methods) (i)HIV inhibitor response f2a(x1 x2) (ii) second De Jongfunction (Rosenbrockrsquos saddle) f2b(x1 x2) (iii) normal-ized lung cancer inhibition response f2c(x1 x2) and (iv)bacterial (S aureus) inhibition response f2d(x1 x2)These functions are shown in Figure 3 For each drugresponse function we searched for the optimal drugresponse using the proposed algorithm starting fromrandomly selected drug concentrations x1 and x2 Theparameter a that controls the randomness of the searchwas set to a = 1 which implies that the algorithm adap-tively determines the search direction (ie how toupdate the current drug concentration) by fully utilizingthe past observations Note that setting the parameter toa = 0 in (7) would make the search completely randomand independent of the past observations at each stepthe concentration of a given drug will be randomlyincreased or decreased with equal probability regardlessof the update rules given in (5) In each search experi-ment the iterations were repeated up to two times thetotal number of possible drug combinations Thisexperiment was repeated 5000 times to obtain reliableresults For comparison we performed similar experi-ments using the Gur Game algorithm [9] and the sto-chastic search algorithm proposed in [11] with a = 1We computed the following two performance metricsthe success rate and the average number of unique drugcombinations that need to be tested The first metric isdefined as the relative proportion of experiments inwhich the algorithm was able to identify a potent drugcombination whose response is within 5 of the maxi-mum response ie f(x1 x2) ge 095 The second metricis defined as the average number of unique drug combi-nations that have to be tested until a potent drug com-bination is identified in case the search is successfulThese performance assessment results are summarizedin Table 1 Note that the Gur Game algorithm has beentested using both the simultaneous update strategy as

well as the sequential update strategy Unlike the ARUalgorithm and the search algorithm proposed in [11]which update one drug at a time (ie ldquosequentialrdquo drugupdate) the original Gur Game algorithm adopted in[9] updates all drugs simultaneously However it is alsopossible to use the sequential update strategy with theGur Game algorithm and we have evaluated both stra-tegies for comparison Table 1 shows that the proposedARU algorithm outperforms the existing algorithms interms of success rate Furthermore when the successrates are comparable the ARU algorithm can in generalidentify an effective drug combination more efficientlyas reflected in the smaller number of unique drug com-binations that need to be tested We can get a morecomplete picture of the efficiency of the ARU algorithmfrom Figure S1 and Figure S2 which respectively showthe distribution of the number of unique drug combina-tions that need to be tested and the distribution of thenumber of iterations that are needed to identify a potentdrug combination (see Additional file 1)

Optimizing multi-drug cocktailsNext we tested the performance of the ARU algorithmfor optimizing multi-drug cocktails that consist of threeto six drugs For this purpose we used the eight hypothe-tical drug response functions that were defined before(see Methods) As in our previous experiments for eachdrug response function we used the proposed ARU algo-rithm (with a = 1) to search for a potent drug combina-tion whose response is within 5 of the maximumresponse (ie f(x) ge 095) In each search experiment westarted from an randomly selected initial concentrationsand continued the search up to 1000 steps for 3 drugs2000 steps for 4 drugs 3000 steps for 5 drugs and 4000steps for 6 drugs This experiment was repeated 5000times to obtain reliable performance assessment resultsThe simulation results are summarized in Table 2 Asshown in this table the proposed algorithm boasts a sig-nificantly higher success rate compared to the Gur Gamealgorithm [9] It also results in either comparable orslightly improved success rate compared to the previousdrug optimization algorithm (a = 1) [11] However we

Table 1 Performance for optimizing the combination of two drugs

Gur Game(simultaneous)

Gur Game (sequential) Previous searchalgorithm [11](a = 1)

ARU algorithm(proposed) (a = 1)

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

successrate

uniquecomb

f2a(x) HIV INHIBITION (M = 80) 97 132 95 172 100 134 100 121

f2b(x) DEJONG (2ND) (M = 441) 9 36 10 45 99 562 99 462

f2c(x) CANCERINHIBITION

(M = 100) 58 113 53 130 98 132 98 124

f2d(x) BACTERIAINHIBITION

(M = 81) 96 59 91 68 100 48 100 45

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 8 of 15

can see that the ARU algorithm clearly outperforms theprevious search algorithm in terms of efficiency asreflected in the significantly smaller number of uniquedrug combinations that need to be tested until an effec-tive combination is identified Figures S3 and S5 showthe distribution of the number of unique drug combina-tions that need to be tested to identify an effective drugcocktail (see Additional file 1) Similarly Figures S4 andS6 show the distribution of the number of search itera-tions that are needed by each algorithm

Drug optimization in the presence of measurement noiseIn order to use a drug optimization algorithm in practi-cal applications the algorithm has to be robust to ran-dom fluctuations in the estimated drug response Toevaluate the robustness of the proposed ARU algorithmwe evaluated its search performance in the presence ofmeasurement noise and compared it with other existingstochastic search algorithms In these experiments weconsidered two different types of search strategies Inthe first search strategy (referred as type-A) when thesearch algorithm happens to revisit a drug combinationthat was previously tested it does not re-evaluate thedrug response and simply uses the previously estimatedvalue On the other hand according to the second strat-egy (referred as type-B) the search algorithm always re-evaluates the drug response even if it revisits a pre-viously evaluated drug combination since the measuredresponse may be different every time due to the randommeasurement noise The first strategy may be usefulwhen the noise level is relatively low in which case thisstrategy may be able to reduce the total number of drugresponse evaluations thereby reducing the overallexperimental cost for identifying a potent drug combi-nation However when the noise level is high thesearch performance may be degraded as the search algo-rithm clings to the past (noisy) response once it hasbeen measured In contrast the second search strategygenerally requires a relatively larger number of drugresponse evaluations but it tends to be more robust to

random fluctuations and noise in the measured drugresponse functionIn order to evaluate the performance of the different

search algorithms in the presence of noise we per-formed similar search experiments as before at three dif-ferent levels of additive noise 2 5 and 8 Moreprecisely we assume that

fobs(x) = ftrue(x) + η

where fobs(x) is the observed drug response ftrue(x) is thetrue response and h is an independent random noise thatis uniformly distributed over (-u u) where u Icirc 002 005008 For each drug response function and a given noiselevel u we tested the performance of both search strate-gies For type-A search we evaluated the success rate andthe average number of unique drug combinations thathave to be tested until a potent drug combination is identi-fied For type-B search we evaluated the success rate andthe average number of iterations instead of the number ofunique drug combinations until an effective combinationis identified This is because in a type-B search the searchalgorithm re-evaluates the drug response even if it revisitsthe same drug combination that was previously tested Thesimulation results are shown in Table 3 4 5 6 7 for drugresponse functions with two to six drugs The parameter awas set to a = 1 for the ARU algorithm as well as the pre-vious search algorithm proposed in [11]As we can see in these Tables measurement noise cer-

tainly affects the overall performance of the ARU algo-rithm where a higher noise tends to reduce the successrate and increase the number of iterations as well as thatof the unique drug combinations to be tested For manydrug response functions considered in our simulationsthe performance degradation is typically not too signifi-cant for the proposed algorithm showing that the ARUalgorithm is relatively robust to measurement noiseHowever we can also observe that the extent of perfor-mance degradation will critically depend on the land-scape of the underlying drug response In most cases the

Table 2 Performance for optimizing multi-drug cocktails

Gur Game (simultaneous) Gur Game (sequential) Previous search algorithm [11](a = 1)

ARU algorithm (proposed)(a = 1)

success rate unique comb success rate unique comb success rate unique comb success rate unique comb

f3a(x) (M = 113) 1 43 2 55 100 1053 100 740

f3b(x) (M = 113) 83 2294 58 2048 100 885 100 794

f4a(x) (M = 114) 20 8239 11 6666 100 1779 100 1368

f4b(x) (M = 114) 52 7067 24 5209 100 1179 100 916

f5a(x) (M = 115) 8 21 2 48 100 1381 100 806

f5b(x) (M = 115) 89 10134 54 9762 100 2529 100 2168

f6a(x) (M = 116) 90 12691 44 12608 100 1919 100 1781

f6b(x) (M = 116) 90 4467 40 10332 100 2381 100 1901

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 9 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f2a(x) (2) A success rateunique comb

97 96 100 100

125 170 133 116

B success rateiterations

97 95 100 100

378 452 258 200

(5) A success rateunique comb

97 96 100 100

126 171 133 118

B success rateiterations

97 95 100 100

380 454 266 202

(8) A success rateunique comb

97 96 100 100

126 170 133 120

B success rateiterations

97 95 100 100

382 454 268 204

f2b(x) (2) A success rateunique comb

10 10 99 99

39 42 570 451

B success rateiterations

10 10 99 99

40 44 1485 1200

(5) A success rateunique comb

9 9 98 98

41 45 629 522

B success rateiterations

9 9 98 99

43 47 1721 1438

(8) A success rateunique comb

8 9 97 98

41 47 662 556

B success rateiterations

9 9 97 98

44 49 1981 1673

f2c(x) (2) A success rateunique comb

61 54 98 98

109 127 131 125

B success rateiterations

60 54 98 98

331 360 359 352

(5) A success rateunique comb

71 66 98 98

114 132 129 124

B success rateiterations

60 54 98 98

332 354 367 360

(8) A success rateunique comb

88 83 98 98

117 134 126 120

B success rateiterations

6 54 98 98

334 343 374 370

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 10 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise (Continued)

f2d(x) (2) A success rateunique comb

100 100 100 100

49 60 46 41

B success rateiterations

96 9 100 100

183 197 82 77

(5) A success rateunique comb

100 100 100 100

49 57 43 41

B success rateiterations

96 91 100 100

184 196 81 75

(8) A success rateunique comb

100 100 100 100

48 56 44 42

B success rateiterations

96 91 100 100

186 196 81 71

Table 4 Performance for optimizing the combination of three drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f3a(x) (2) A success rateunique comb

1 3 99 100

24 73 1106 773

B success rateiterations

1 3 99 100

28 109 2011 1396

(5) A success rateunique comb

1 3 99 100

24 74 1117 784

B success rateiterations

1 3 99 100

25 101 2016 1440

(8) A success rateunique comb

1 3 99 100

25 77 1133 805

B success rateiterations

1 3 99 100

23 94 2109 1517

f3b(x) (2) A success rateunique comb

86 69 99 99

2243 2016 1108 933

B success rateiterations

83 59 99 99

3671 4191 2115 2053

(5) A success rateunique comb

89 72 99 99

2223 2016 1166 1062

B success rateiterations

83 59 98 98

3593 4399 2255 2229

(8) A success rateunique comb

90 74 97 98

2259 2009 1264 1143

B success rateiterations

82 60 97 98

3594 4313 2491 2468

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 11 of 15

Table 5 Performance for optimizing the combination of four drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f4a(x) (2) A success rateunique comb

21 13 96 98

7989 7117 3939 3274

B success rateiterations

21 12 96 97

9419 10321 5583 4523

(5) A success rateunique comb

21 14 90 95

8163 6536 4731 3983

B success rateiterations

21 13 90 95

8956 10229 6754 5812

(8) A success rateunique comb

24 14 85 95

8587 6816 5057 4336

B success rateiterations

23 13 84 92

9971 10089 7208 6485

f4b(x) (2) A success rateunique comb

62 41 100 100

6345 5231 1380 1031

B success rateiterations

51 26 100 100

9324 9030 2369 1828

(5) A success rateunique comb

75 68 100 100

6102 4680 2311 1509

B success rateiterations

50 25 100 100

8559 9212 4110 2588

(8) A success rateunique comb

86 82 98 100

5258 4301 3142 2159

B success rateiterations

50 24 94 100

8353 9792 6020 3938

Table 6 Performance for optimizing the combination of five drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f5a(x) (2) A success rateunique comb

8 9 100 100

21 44 1393 1225

B success rateiterations

9 11 100 100

21 60 1724 1549

(5) A success rateunique comb

9 11 100 100

39 79 1421 1291

B success rateiterations

9 12 100 100

1083 386 1771 1556

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 12 of 15

Table 6 Performance for optimizing the combination of five drugs in the presence of noise (Continued)

(8) A success rateunique comb

10 13 100 100

71 202 1445 1314

B success rateiterations

9 13 100 100

1914 708 1823 1565

f5b(x) (2) A success rateunique comb

89 55 100 100

9179 10263 4071 3439

B success rateiterations

90 55 100 100

9998 13251 5165 4443

(5) A success rateunique comb

90 59 97 98

9328 10027 5077 4636

B success rateiterations

90 56 99 99

10047 13327 6569 5624

(8) A success rateunique comb

91 59 97 98

9595 9712 5788 5348

B success rateiterations

90 56 99 99

10152 13410 7350 6686

Table 7 Performance for optimizing the combination of six drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f6a(x) (2) A success rateunique comb

91 43 100 100

12800 12141 4768 4326

B success rateiterations

90 43 100 100

13526 16626 5314 5036

(5) A success rateunique comb

90 44 99 99

12623 12472 6210 5983

B success rateiterations

90 45 100 100

13968 16759 7637 7361

(8) A success rateunique comb

90 46 98 98

12047 13024 7232 6988

B success rateiterations

90 46 98 98

14122 16819 8752 8343

f6b(x) (2) A success rateunique comb

91 43 100 100

5099 9710 3414 2377

B success rateiterations

94 44 100 100

12407 16460 4365 2935

(5) A success rateunique comb

90 42 100 100

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 13 of 15

ARU algorithm continued to substantially outperformother stochastic search algorithms [911] demonstratingthat it is better suited for practical drug optimizationapplicationsOne interesting observation is that the performance of

the Gur Game algorithm is typically not very sensitiveto measurement noise In fact in some cases its perfor-mance even improves as the noise level goes up Themain reason for this phenomenon is as follows As dis-cussed earlier the Gur Game algorithm does not adaptto the observed drug response function and for this rea-son its overall performance crucially depends onwhether or not its predetermined FSA matches the drugresponse function at hand As a result if the FSA doesnot match the original drug response function well iro-nically enough the measurement noise may perturb thesearch process in such a way that improves the overallperformance In this sense the fact that the Gur Gamealgorithm is not very sensitive to measurement noisereflects its inaptitude for handling various types of drugresponse functions rather than its robustness to randomfluctuations and noise in the measured drug response

ConclusionsIn this paper we proposed a novel stochastic searchalgorithm called the adaptive reference update (ARU)algorithm which can be effectively used for optimizingthe composition of combinatory drugs The proposedalgorithm intelligently utilizes the drug response valuesobserved in the past to reliably predict how to benefi-cially update the drug concentrations to improve thedrug response As we demonstrated throughout thispaper the proposed algorithm addresses several short-comings of previous drug optimization algorithms[911] thereby improving the overall search perfor-mance Numerical experiments based on various typesof multi-drug response functions show that the ARUalgorithm results in a higher success rate (ie higherprobability of identifying a potent drug combination)while requiring significantly fewer drug response evalua-tions Furthermore the proposed algorithm is robust to

random measurement noise where its search perfor-mance is not substantially affected in the presence ofnoise and degrades gracefully as the noise levelincreases Throughout our experiments we used a = 1for the ARU algorithm as well as the previous searchalgorithm [11] As discussed earlier the parameter acontrols the randomness of the search by determininghow much weight we should give to the drug responsevalues that we observed in the past In general unlessthe observations are very noisy or the underlying drugresponse function is assumed to be extremely nonlinearit would be best to set the parameter to the largestallowed value (ie a = 1) so that we fully utilize thepast observations for making our best informed guessabout the beneficial drug update strategy For compari-son we also repeated our simulations using a = 05 anda = 075 whose results are summarized in Table S1 -Table S7 (see Additional file 1) We can see from theseresults that a = 1 indeed leads to the best performancefor the drug response functions and the noise levels wehave considered in this paper

Additional material

Additional file 1 Performance of the ARU algorithm Furtherperformance evaluation results of the proposed Adaptive ReferenceUpdate (ARU) algorithm

AcknowledgementsBased on ldquoEfficient combinatorial drug optimization through stochasticsearchrdquo by Mansuck Kim and Byung-Jun Yoon which appeared in GenomicSignal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop oncopy 2011 IEEE [19]This work was supported in part by the National Science Foundationthrough NSF Award CCF-1149544This article has been published as part of BMC Genomics Volume 13Supplement 6 2012 Selected articles from the IEEE International Workshopon Genomic Signal Processing and Statistics (GENSIPS) 2011 The fullcontents of the supplement are available online at httpwwwbiomedcentralcombmcgenomicssupplements13S6

Authorsrsquo contributionsConceived and developed the algorithm MK BJY Performed theexperiments MK Analyzed the data and wrote the paper MK BJY

Table 7 Performance for optimizing the combination of six drugs in the presence of noise (Continued)

4738 9707 3499 2798

B success rateiterations

94 44 100 100

12786 17049 4808 3246

(8) A success rateunique comb

89 42 100 100

4544 9699 4574 3532

B success rateiterations

94 44 100 100

13331 17755 5453 3915

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 14 of 15

can see that the ARU algorithm clearly outperforms theprevious search algorithm in terms of efficiency asreflected in the significantly smaller number of uniquedrug combinations that need to be tested until an effec-tive combination is identified Figures S3 and S5 showthe distribution of the number of unique drug combina-tions that need to be tested to identify an effective drugcocktail (see Additional file 1) Similarly Figures S4 andS6 show the distribution of the number of search itera-tions that are needed by each algorithm

Drug optimization in the presence of measurement noiseIn order to use a drug optimization algorithm in practi-cal applications the algorithm has to be robust to ran-dom fluctuations in the estimated drug response Toevaluate the robustness of the proposed ARU algorithmwe evaluated its search performance in the presence ofmeasurement noise and compared it with other existingstochastic search algorithms In these experiments weconsidered two different types of search strategies Inthe first search strategy (referred as type-A) when thesearch algorithm happens to revisit a drug combinationthat was previously tested it does not re-evaluate thedrug response and simply uses the previously estimatedvalue On the other hand according to the second strat-egy (referred as type-B) the search algorithm always re-evaluates the drug response even if it revisits a pre-viously evaluated drug combination since the measuredresponse may be different every time due to the randommeasurement noise The first strategy may be usefulwhen the noise level is relatively low in which case thisstrategy may be able to reduce the total number of drugresponse evaluations thereby reducing the overallexperimental cost for identifying a potent drug combi-nation However when the noise level is high thesearch performance may be degraded as the search algo-rithm clings to the past (noisy) response once it hasbeen measured In contrast the second search strategygenerally requires a relatively larger number of drugresponse evaluations but it tends to be more robust to

random fluctuations and noise in the measured drugresponse functionIn order to evaluate the performance of the different

search algorithms in the presence of noise we per-formed similar search experiments as before at three dif-ferent levels of additive noise 2 5 and 8 Moreprecisely we assume that

fobs(x) = ftrue(x) + η

where fobs(x) is the observed drug response ftrue(x) is thetrue response and h is an independent random noise thatis uniformly distributed over (-u u) where u Icirc 002 005008 For each drug response function and a given noiselevel u we tested the performance of both search strate-gies For type-A search we evaluated the success rate andthe average number of unique drug combinations thathave to be tested until a potent drug combination is identi-fied For type-B search we evaluated the success rate andthe average number of iterations instead of the number ofunique drug combinations until an effective combinationis identified This is because in a type-B search the searchalgorithm re-evaluates the drug response even if it revisitsthe same drug combination that was previously tested Thesimulation results are shown in Table 3 4 5 6 7 for drugresponse functions with two to six drugs The parameter awas set to a = 1 for the ARU algorithm as well as the pre-vious search algorithm proposed in [11]As we can see in these Tables measurement noise cer-

tainly affects the overall performance of the ARU algo-rithm where a higher noise tends to reduce the successrate and increase the number of iterations as well as thatof the unique drug combinations to be tested For manydrug response functions considered in our simulationsthe performance degradation is typically not too signifi-cant for the proposed algorithm showing that the ARUalgorithm is relatively robust to measurement noiseHowever we can also observe that the extent of perfor-mance degradation will critically depend on the land-scape of the underlying drug response In most cases the

Table 2 Performance for optimizing multi-drug cocktails

Gur Game (simultaneous) Gur Game (sequential) Previous search algorithm [11](a = 1)

ARU algorithm (proposed)(a = 1)

success rate unique comb success rate unique comb success rate unique comb success rate unique comb

f3a(x) (M = 113) 1 43 2 55 100 1053 100 740

f3b(x) (M = 113) 83 2294 58 2048 100 885 100 794

f4a(x) (M = 114) 20 8239 11 6666 100 1779 100 1368

f4b(x) (M = 114) 52 7067 24 5209 100 1179 100 916

f5a(x) (M = 115) 8 21 2 48 100 1381 100 806

f5b(x) (M = 115) 89 10134 54 9762 100 2529 100 2168

f6a(x) (M = 116) 90 12691 44 12608 100 1919 100 1781

f6b(x) (M = 116) 90 4467 40 10332 100 2381 100 1901

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 9 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f2a(x) (2) A success rateunique comb

97 96 100 100

125 170 133 116

B success rateiterations

97 95 100 100

378 452 258 200

(5) A success rateunique comb

97 96 100 100

126 171 133 118

B success rateiterations

97 95 100 100

380 454 266 202

(8) A success rateunique comb

97 96 100 100

126 170 133 120

B success rateiterations

97 95 100 100

382 454 268 204

f2b(x) (2) A success rateunique comb

10 10 99 99

39 42 570 451

B success rateiterations

10 10 99 99

40 44 1485 1200

(5) A success rateunique comb

9 9 98 98

41 45 629 522

B success rateiterations

9 9 98 99

43 47 1721 1438

(8) A success rateunique comb

8 9 97 98

41 47 662 556

B success rateiterations

9 9 97 98

44 49 1981 1673

f2c(x) (2) A success rateunique comb

61 54 98 98

109 127 131 125

B success rateiterations

60 54 98 98

331 360 359 352

(5) A success rateunique comb

71 66 98 98

114 132 129 124

B success rateiterations

60 54 98 98

332 354 367 360

(8) A success rateunique comb

88 83 98 98

117 134 126 120

B success rateiterations

6 54 98 98

334 343 374 370

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 10 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise (Continued)

f2d(x) (2) A success rateunique comb

100 100 100 100

49 60 46 41

B success rateiterations

96 9 100 100

183 197 82 77

(5) A success rateunique comb

100 100 100 100

49 57 43 41

B success rateiterations

96 91 100 100

184 196 81 75

(8) A success rateunique comb

100 100 100 100

48 56 44 42

B success rateiterations

96 91 100 100

186 196 81 71

Table 4 Performance for optimizing the combination of three drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f3a(x) (2) A success rateunique comb

1 3 99 100

24 73 1106 773

B success rateiterations

1 3 99 100

28 109 2011 1396

(5) A success rateunique comb

1 3 99 100

24 74 1117 784

B success rateiterations

1 3 99 100

25 101 2016 1440

(8) A success rateunique comb

1 3 99 100

25 77 1133 805

B success rateiterations

1 3 99 100

23 94 2109 1517

f3b(x) (2) A success rateunique comb

86 69 99 99

2243 2016 1108 933

B success rateiterations

83 59 99 99

3671 4191 2115 2053

(5) A success rateunique comb

89 72 99 99

2223 2016 1166 1062

B success rateiterations

83 59 98 98

3593 4399 2255 2229

(8) A success rateunique comb

90 74 97 98

2259 2009 1264 1143

B success rateiterations

82 60 97 98

3594 4313 2491 2468

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 11 of 15

Table 5 Performance for optimizing the combination of four drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f4a(x) (2) A success rateunique comb

21 13 96 98

7989 7117 3939 3274

B success rateiterations

21 12 96 97

9419 10321 5583 4523

(5) A success rateunique comb

21 14 90 95

8163 6536 4731 3983

B success rateiterations

21 13 90 95

8956 10229 6754 5812

(8) A success rateunique comb

24 14 85 95

8587 6816 5057 4336

B success rateiterations

23 13 84 92

9971 10089 7208 6485

f4b(x) (2) A success rateunique comb

62 41 100 100

6345 5231 1380 1031

B success rateiterations

51 26 100 100

9324 9030 2369 1828

(5) A success rateunique comb

75 68 100 100

6102 4680 2311 1509

B success rateiterations

50 25 100 100

8559 9212 4110 2588

(8) A success rateunique comb

86 82 98 100

5258 4301 3142 2159

B success rateiterations

50 24 94 100

8353 9792 6020 3938

Table 6 Performance for optimizing the combination of five drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f5a(x) (2) A success rateunique comb

8 9 100 100

21 44 1393 1225

B success rateiterations

9 11 100 100

21 60 1724 1549

(5) A success rateunique comb

9 11 100 100

39 79 1421 1291

B success rateiterations

9 12 100 100

1083 386 1771 1556

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 12 of 15

Table 6 Performance for optimizing the combination of five drugs in the presence of noise (Continued)

(8) A success rateunique comb

10 13 100 100

71 202 1445 1314

B success rateiterations

9 13 100 100

1914 708 1823 1565

f5b(x) (2) A success rateunique comb

89 55 100 100

9179 10263 4071 3439

B success rateiterations

90 55 100 100

9998 13251 5165 4443

(5) A success rateunique comb

90 59 97 98

9328 10027 5077 4636

B success rateiterations

90 56 99 99

10047 13327 6569 5624

(8) A success rateunique comb

91 59 97 98

9595 9712 5788 5348

B success rateiterations

90 56 99 99

10152 13410 7350 6686

Table 7 Performance for optimizing the combination of six drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f6a(x) (2) A success rateunique comb

91 43 100 100

12800 12141 4768 4326

B success rateiterations

90 43 100 100

13526 16626 5314 5036

(5) A success rateunique comb

90 44 99 99

12623 12472 6210 5983

B success rateiterations

90 45 100 100

13968 16759 7637 7361

(8) A success rateunique comb

90 46 98 98

12047 13024 7232 6988

B success rateiterations

90 46 98 98

14122 16819 8752 8343

f6b(x) (2) A success rateunique comb

91 43 100 100

5099 9710 3414 2377

B success rateiterations

94 44 100 100

12407 16460 4365 2935

(5) A success rateunique comb

90 42 100 100

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 13 of 15

ARU algorithm continued to substantially outperformother stochastic search algorithms [911] demonstratingthat it is better suited for practical drug optimizationapplicationsOne interesting observation is that the performance of

the Gur Game algorithm is typically not very sensitiveto measurement noise In fact in some cases its perfor-mance even improves as the noise level goes up Themain reason for this phenomenon is as follows As dis-cussed earlier the Gur Game algorithm does not adaptto the observed drug response function and for this rea-son its overall performance crucially depends onwhether or not its predetermined FSA matches the drugresponse function at hand As a result if the FSA doesnot match the original drug response function well iro-nically enough the measurement noise may perturb thesearch process in such a way that improves the overallperformance In this sense the fact that the Gur Gamealgorithm is not very sensitive to measurement noisereflects its inaptitude for handling various types of drugresponse functions rather than its robustness to randomfluctuations and noise in the measured drug response

ConclusionsIn this paper we proposed a novel stochastic searchalgorithm called the adaptive reference update (ARU)algorithm which can be effectively used for optimizingthe composition of combinatory drugs The proposedalgorithm intelligently utilizes the drug response valuesobserved in the past to reliably predict how to benefi-cially update the drug concentrations to improve thedrug response As we demonstrated throughout thispaper the proposed algorithm addresses several short-comings of previous drug optimization algorithms[911] thereby improving the overall search perfor-mance Numerical experiments based on various typesof multi-drug response functions show that the ARUalgorithm results in a higher success rate (ie higherprobability of identifying a potent drug combination)while requiring significantly fewer drug response evalua-tions Furthermore the proposed algorithm is robust to

random measurement noise where its search perfor-mance is not substantially affected in the presence ofnoise and degrades gracefully as the noise levelincreases Throughout our experiments we used a = 1for the ARU algorithm as well as the previous searchalgorithm [11] As discussed earlier the parameter acontrols the randomness of the search by determininghow much weight we should give to the drug responsevalues that we observed in the past In general unlessthe observations are very noisy or the underlying drugresponse function is assumed to be extremely nonlinearit would be best to set the parameter to the largestallowed value (ie a = 1) so that we fully utilize thepast observations for making our best informed guessabout the beneficial drug update strategy For compari-son we also repeated our simulations using a = 05 anda = 075 whose results are summarized in Table S1 -Table S7 (see Additional file 1) We can see from theseresults that a = 1 indeed leads to the best performancefor the drug response functions and the noise levels wehave considered in this paper

Additional material

Additional file 1 Performance of the ARU algorithm Furtherperformance evaluation results of the proposed Adaptive ReferenceUpdate (ARU) algorithm

AcknowledgementsBased on ldquoEfficient combinatorial drug optimization through stochasticsearchrdquo by Mansuck Kim and Byung-Jun Yoon which appeared in GenomicSignal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop oncopy 2011 IEEE [19]This work was supported in part by the National Science Foundationthrough NSF Award CCF-1149544This article has been published as part of BMC Genomics Volume 13Supplement 6 2012 Selected articles from the IEEE International Workshopon Genomic Signal Processing and Statistics (GENSIPS) 2011 The fullcontents of the supplement are available online at httpwwwbiomedcentralcombmcgenomicssupplements13S6

Authorsrsquo contributionsConceived and developed the algorithm MK BJY Performed theexperiments MK Analyzed the data and wrote the paper MK BJY

Table 7 Performance for optimizing the combination of six drugs in the presence of noise (Continued)

4738 9707 3499 2798

B success rateiterations

94 44 100 100

12786 17049 4808 3246

(8) A success rateunique comb

89 42 100 100

4544 9699 4574 3532

B success rateiterations

94 44 100 100

13331 17755 5453 3915

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 14 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f2a(x) (2) A success rateunique comb

97 96 100 100

125 170 133 116

B success rateiterations

97 95 100 100

378 452 258 200

(5) A success rateunique comb

97 96 100 100

126 171 133 118

B success rateiterations

97 95 100 100

380 454 266 202

(8) A success rateunique comb

97 96 100 100

126 170 133 120

B success rateiterations

97 95 100 100

382 454 268 204

f2b(x) (2) A success rateunique comb

10 10 99 99

39 42 570 451

B success rateiterations

10 10 99 99

40 44 1485 1200

(5) A success rateunique comb

9 9 98 98

41 45 629 522

B success rateiterations

9 9 98 99

43 47 1721 1438

(8) A success rateunique comb

8 9 97 98

41 47 662 556

B success rateiterations

9 9 97 98

44 49 1981 1673

f2c(x) (2) A success rateunique comb

61 54 98 98

109 127 131 125

B success rateiterations

60 54 98 98

331 360 359 352

(5) A success rateunique comb

71 66 98 98

114 132 129 124

B success rateiterations

60 54 98 98

332 354 367 360

(8) A success rateunique comb

88 83 98 98

117 134 126 120

B success rateiterations

6 54 98 98

334 343 374 370

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 10 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise (Continued)

f2d(x) (2) A success rateunique comb

100 100 100 100

49 60 46 41

B success rateiterations

96 9 100 100

183 197 82 77

(5) A success rateunique comb

100 100 100 100

49 57 43 41

B success rateiterations

96 91 100 100

184 196 81 75

(8) A success rateunique comb

100 100 100 100

48 56 44 42

B success rateiterations

96 91 100 100

186 196 81 71

Table 4 Performance for optimizing the combination of three drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f3a(x) (2) A success rateunique comb

1 3 99 100

24 73 1106 773

B success rateiterations

1 3 99 100

28 109 2011 1396

(5) A success rateunique comb

1 3 99 100

24 74 1117 784

B success rateiterations

1 3 99 100

25 101 2016 1440

(8) A success rateunique comb

1 3 99 100

25 77 1133 805

B success rateiterations

1 3 99 100

23 94 2109 1517

f3b(x) (2) A success rateunique comb

86 69 99 99

2243 2016 1108 933

B success rateiterations

83 59 99 99

3671 4191 2115 2053

(5) A success rateunique comb

89 72 99 99

2223 2016 1166 1062

B success rateiterations

83 59 98 98

3593 4399 2255 2229

(8) A success rateunique comb

90 74 97 98

2259 2009 1264 1143

B success rateiterations

82 60 97 98

3594 4313 2491 2468

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 11 of 15

Table 5 Performance for optimizing the combination of four drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f4a(x) (2) A success rateunique comb

21 13 96 98

7989 7117 3939 3274

B success rateiterations

21 12 96 97

9419 10321 5583 4523

(5) A success rateunique comb

21 14 90 95

8163 6536 4731 3983

B success rateiterations

21 13 90 95

8956 10229 6754 5812

(8) A success rateunique comb

24 14 85 95

8587 6816 5057 4336

B success rateiterations

23 13 84 92

9971 10089 7208 6485

f4b(x) (2) A success rateunique comb

62 41 100 100

6345 5231 1380 1031

B success rateiterations

51 26 100 100

9324 9030 2369 1828

(5) A success rateunique comb

75 68 100 100

6102 4680 2311 1509

B success rateiterations

50 25 100 100

8559 9212 4110 2588

(8) A success rateunique comb

86 82 98 100

5258 4301 3142 2159

B success rateiterations

50 24 94 100

8353 9792 6020 3938

Table 6 Performance for optimizing the combination of five drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f5a(x) (2) A success rateunique comb

8 9 100 100

21 44 1393 1225

B success rateiterations

9 11 100 100

21 60 1724 1549

(5) A success rateunique comb

9 11 100 100

39 79 1421 1291

B success rateiterations

9 12 100 100

1083 386 1771 1556

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 12 of 15

Table 6 Performance for optimizing the combination of five drugs in the presence of noise (Continued)

(8) A success rateunique comb

10 13 100 100

71 202 1445 1314

B success rateiterations

9 13 100 100

1914 708 1823 1565

f5b(x) (2) A success rateunique comb

89 55 100 100

9179 10263 4071 3439

B success rateiterations

90 55 100 100

9998 13251 5165 4443

(5) A success rateunique comb

90 59 97 98

9328 10027 5077 4636

B success rateiterations

90 56 99 99

10047 13327 6569 5624

(8) A success rateunique comb

91 59 97 98

9595 9712 5788 5348

B success rateiterations

90 56 99 99

10152 13410 7350 6686

Table 7 Performance for optimizing the combination of six drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f6a(x) (2) A success rateunique comb

91 43 100 100

12800 12141 4768 4326

B success rateiterations

90 43 100 100

13526 16626 5314 5036

(5) A success rateunique comb

90 44 99 99

12623 12472 6210 5983

B success rateiterations

90 45 100 100

13968 16759 7637 7361

(8) A success rateunique comb

90 46 98 98

12047 13024 7232 6988

B success rateiterations

90 46 98 98

14122 16819 8752 8343

f6b(x) (2) A success rateunique comb

91 43 100 100

5099 9710 3414 2377

B success rateiterations

94 44 100 100

12407 16460 4365 2935

(5) A success rateunique comb

90 42 100 100

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 13 of 15

ARU algorithm continued to substantially outperformother stochastic search algorithms [911] demonstratingthat it is better suited for practical drug optimizationapplicationsOne interesting observation is that the performance of

the Gur Game algorithm is typically not very sensitiveto measurement noise In fact in some cases its perfor-mance even improves as the noise level goes up Themain reason for this phenomenon is as follows As dis-cussed earlier the Gur Game algorithm does not adaptto the observed drug response function and for this rea-son its overall performance crucially depends onwhether or not its predetermined FSA matches the drugresponse function at hand As a result if the FSA doesnot match the original drug response function well iro-nically enough the measurement noise may perturb thesearch process in such a way that improves the overallperformance In this sense the fact that the Gur Gamealgorithm is not very sensitive to measurement noisereflects its inaptitude for handling various types of drugresponse functions rather than its robustness to randomfluctuations and noise in the measured drug response

ConclusionsIn this paper we proposed a novel stochastic searchalgorithm called the adaptive reference update (ARU)algorithm which can be effectively used for optimizingthe composition of combinatory drugs The proposedalgorithm intelligently utilizes the drug response valuesobserved in the past to reliably predict how to benefi-cially update the drug concentrations to improve thedrug response As we demonstrated throughout thispaper the proposed algorithm addresses several short-comings of previous drug optimization algorithms[911] thereby improving the overall search perfor-mance Numerical experiments based on various typesof multi-drug response functions show that the ARUalgorithm results in a higher success rate (ie higherprobability of identifying a potent drug combination)while requiring significantly fewer drug response evalua-tions Furthermore the proposed algorithm is robust to

random measurement noise where its search perfor-mance is not substantially affected in the presence ofnoise and degrades gracefully as the noise levelincreases Throughout our experiments we used a = 1for the ARU algorithm as well as the previous searchalgorithm [11] As discussed earlier the parameter acontrols the randomness of the search by determininghow much weight we should give to the drug responsevalues that we observed in the past In general unlessthe observations are very noisy or the underlying drugresponse function is assumed to be extremely nonlinearit would be best to set the parameter to the largestallowed value (ie a = 1) so that we fully utilize thepast observations for making our best informed guessabout the beneficial drug update strategy For compari-son we also repeated our simulations using a = 05 anda = 075 whose results are summarized in Table S1 -Table S7 (see Additional file 1) We can see from theseresults that a = 1 indeed leads to the best performancefor the drug response functions and the noise levels wehave considered in this paper

Additional material

Additional file 1 Performance of the ARU algorithm Furtherperformance evaluation results of the proposed Adaptive ReferenceUpdate (ARU) algorithm

AcknowledgementsBased on ldquoEfficient combinatorial drug optimization through stochasticsearchrdquo by Mansuck Kim and Byung-Jun Yoon which appeared in GenomicSignal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop oncopy 2011 IEEE [19]This work was supported in part by the National Science Foundationthrough NSF Award CCF-1149544This article has been published as part of BMC Genomics Volume 13Supplement 6 2012 Selected articles from the IEEE International Workshopon Genomic Signal Processing and Statistics (GENSIPS) 2011 The fullcontents of the supplement are available online at httpwwwbiomedcentralcombmcgenomicssupplements13S6

Authorsrsquo contributionsConceived and developed the algorithm MK BJY Performed theexperiments MK Analyzed the data and wrote the paper MK BJY

Table 7 Performance for optimizing the combination of six drugs in the presence of noise (Continued)

4738 9707 3499 2798

B success rateiterations

94 44 100 100

12786 17049 4808 3246

(8) A success rateunique comb

89 42 100 100

4544 9699 4574 3532

B success rateiterations

94 44 100 100

13331 17755 5453 3915

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 14 of 15

Table 3 Performance for optimizing the combination of two drugs in the presence of noise (Continued)

f2d(x) (2) A success rateunique comb

100 100 100 100

49 60 46 41

B success rateiterations

96 9 100 100

183 197 82 77

(5) A success rateunique comb

100 100 100 100

49 57 43 41

B success rateiterations

96 91 100 100

184 196 81 75

(8) A success rateunique comb

100 100 100 100

48 56 44 42

B success rateiterations

96 91 100 100

186 196 81 71

Table 4 Performance for optimizing the combination of three drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f3a(x) (2) A success rateunique comb

1 3 99 100

24 73 1106 773

B success rateiterations

1 3 99 100

28 109 2011 1396

(5) A success rateunique comb

1 3 99 100

24 74 1117 784

B success rateiterations

1 3 99 100

25 101 2016 1440

(8) A success rateunique comb

1 3 99 100

25 77 1133 805

B success rateiterations

1 3 99 100

23 94 2109 1517

f3b(x) (2) A success rateunique comb

86 69 99 99

2243 2016 1108 933

B success rateiterations

83 59 99 99

3671 4191 2115 2053

(5) A success rateunique comb

89 72 99 99

2223 2016 1166 1062

B success rateiterations

83 59 98 98

3593 4399 2255 2229

(8) A success rateunique comb

90 74 97 98

2259 2009 1264 1143

B success rateiterations

82 60 97 98

3594 4313 2491 2468

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 11 of 15

Table 5 Performance for optimizing the combination of four drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f4a(x) (2) A success rateunique comb

21 13 96 98

7989 7117 3939 3274

B success rateiterations

21 12 96 97

9419 10321 5583 4523

(5) A success rateunique comb

21 14 90 95

8163 6536 4731 3983

B success rateiterations

21 13 90 95

8956 10229 6754 5812

(8) A success rateunique comb

24 14 85 95

8587 6816 5057 4336

B success rateiterations

23 13 84 92

9971 10089 7208 6485

f4b(x) (2) A success rateunique comb

62 41 100 100

6345 5231 1380 1031

B success rateiterations

51 26 100 100

9324 9030 2369 1828

(5) A success rateunique comb

75 68 100 100

6102 4680 2311 1509

B success rateiterations

50 25 100 100

8559 9212 4110 2588

(8) A success rateunique comb

86 82 98 100

5258 4301 3142 2159

B success rateiterations

50 24 94 100

8353 9792 6020 3938

Table 6 Performance for optimizing the combination of five drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f5a(x) (2) A success rateunique comb

8 9 100 100

21 44 1393 1225

B success rateiterations

9 11 100 100

21 60 1724 1549

(5) A success rateunique comb

9 11 100 100

39 79 1421 1291

B success rateiterations

9 12 100 100

1083 386 1771 1556

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 12 of 15

Table 6 Performance for optimizing the combination of five drugs in the presence of noise (Continued)

(8) A success rateunique comb

10 13 100 100

71 202 1445 1314

B success rateiterations

9 13 100 100

1914 708 1823 1565

f5b(x) (2) A success rateunique comb

89 55 100 100

9179 10263 4071 3439

B success rateiterations

90 55 100 100

9998 13251 5165 4443

(5) A success rateunique comb

90 59 97 98

9328 10027 5077 4636

B success rateiterations

90 56 99 99

10047 13327 6569 5624

(8) A success rateunique comb

91 59 97 98

9595 9712 5788 5348

B success rateiterations

90 56 99 99

10152 13410 7350 6686

Table 7 Performance for optimizing the combination of six drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f6a(x) (2) A success rateunique comb

91 43 100 100

12800 12141 4768 4326

B success rateiterations

90 43 100 100

13526 16626 5314 5036

(5) A success rateunique comb

90 44 99 99

12623 12472 6210 5983

B success rateiterations

90 45 100 100

13968 16759 7637 7361

(8) A success rateunique comb

90 46 98 98

12047 13024 7232 6988

B success rateiterations

90 46 98 98

14122 16819 8752 8343

f6b(x) (2) A success rateunique comb

91 43 100 100

5099 9710 3414 2377

B success rateiterations

94 44 100 100

12407 16460 4365 2935

(5) A success rateunique comb

90 42 100 100

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 13 of 15

ARU algorithm continued to substantially outperformother stochastic search algorithms [911] demonstratingthat it is better suited for practical drug optimizationapplicationsOne interesting observation is that the performance of

the Gur Game algorithm is typically not very sensitiveto measurement noise In fact in some cases its perfor-mance even improves as the noise level goes up Themain reason for this phenomenon is as follows As dis-cussed earlier the Gur Game algorithm does not adaptto the observed drug response function and for this rea-son its overall performance crucially depends onwhether or not its predetermined FSA matches the drugresponse function at hand As a result if the FSA doesnot match the original drug response function well iro-nically enough the measurement noise may perturb thesearch process in such a way that improves the overallperformance In this sense the fact that the Gur Gamealgorithm is not very sensitive to measurement noisereflects its inaptitude for handling various types of drugresponse functions rather than its robustness to randomfluctuations and noise in the measured drug response

ConclusionsIn this paper we proposed a novel stochastic searchalgorithm called the adaptive reference update (ARU)algorithm which can be effectively used for optimizingthe composition of combinatory drugs The proposedalgorithm intelligently utilizes the drug response valuesobserved in the past to reliably predict how to benefi-cially update the drug concentrations to improve thedrug response As we demonstrated throughout thispaper the proposed algorithm addresses several short-comings of previous drug optimization algorithms[911] thereby improving the overall search perfor-mance Numerical experiments based on various typesof multi-drug response functions show that the ARUalgorithm results in a higher success rate (ie higherprobability of identifying a potent drug combination)while requiring significantly fewer drug response evalua-tions Furthermore the proposed algorithm is robust to

random measurement noise where its search perfor-mance is not substantially affected in the presence ofnoise and degrades gracefully as the noise levelincreases Throughout our experiments we used a = 1for the ARU algorithm as well as the previous searchalgorithm [11] As discussed earlier the parameter acontrols the randomness of the search by determininghow much weight we should give to the drug responsevalues that we observed in the past In general unlessthe observations are very noisy or the underlying drugresponse function is assumed to be extremely nonlinearit would be best to set the parameter to the largestallowed value (ie a = 1) so that we fully utilize thepast observations for making our best informed guessabout the beneficial drug update strategy For compari-son we also repeated our simulations using a = 05 anda = 075 whose results are summarized in Table S1 -Table S7 (see Additional file 1) We can see from theseresults that a = 1 indeed leads to the best performancefor the drug response functions and the noise levels wehave considered in this paper

Additional material

Additional file 1 Performance of the ARU algorithm Furtherperformance evaluation results of the proposed Adaptive ReferenceUpdate (ARU) algorithm

AcknowledgementsBased on ldquoEfficient combinatorial drug optimization through stochasticsearchrdquo by Mansuck Kim and Byung-Jun Yoon which appeared in GenomicSignal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop oncopy 2011 IEEE [19]This work was supported in part by the National Science Foundationthrough NSF Award CCF-1149544This article has been published as part of BMC Genomics Volume 13Supplement 6 2012 Selected articles from the IEEE International Workshopon Genomic Signal Processing and Statistics (GENSIPS) 2011 The fullcontents of the supplement are available online at httpwwwbiomedcentralcombmcgenomicssupplements13S6

Authorsrsquo contributionsConceived and developed the algorithm MK BJY Performed theexperiments MK Analyzed the data and wrote the paper MK BJY

Table 7 Performance for optimizing the combination of six drugs in the presence of noise (Continued)

4738 9707 3499 2798

B success rateiterations

94 44 100 100

12786 17049 4808 3246

(8) A success rateunique comb

89 42 100 100

4544 9699 4574 3532

B success rateiterations

94 44 100 100

13331 17755 5453 3915

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 14 of 15

Table 5 Performance for optimizing the combination of four drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f4a(x) (2) A success rateunique comb

21 13 96 98

7989 7117 3939 3274

B success rateiterations

21 12 96 97

9419 10321 5583 4523

(5) A success rateunique comb

21 14 90 95

8163 6536 4731 3983

B success rateiterations

21 13 90 95

8956 10229 6754 5812

(8) A success rateunique comb

24 14 85 95

8587 6816 5057 4336

B success rateiterations

23 13 84 92

9971 10089 7208 6485

f4b(x) (2) A success rateunique comb

62 41 100 100

6345 5231 1380 1031

B success rateiterations

51 26 100 100

9324 9030 2369 1828

(5) A success rateunique comb

75 68 100 100

6102 4680 2311 1509

B success rateiterations

50 25 100 100

8559 9212 4110 2588

(8) A success rateunique comb

86 82 98 100

5258 4301 3142 2159

B success rateiterations

50 24 94 100

8353 9792 6020 3938

Table 6 Performance for optimizing the combination of five drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f5a(x) (2) A success rateunique comb

8 9 100 100

21 44 1393 1225

B success rateiterations

9 11 100 100

21 60 1724 1549

(5) A success rateunique comb

9 11 100 100

39 79 1421 1291

B success rateiterations

9 12 100 100

1083 386 1771 1556

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 12 of 15

Table 6 Performance for optimizing the combination of five drugs in the presence of noise (Continued)

(8) A success rateunique comb

10 13 100 100

71 202 1445 1314

B success rateiterations

9 13 100 100

1914 708 1823 1565

f5b(x) (2) A success rateunique comb

89 55 100 100

9179 10263 4071 3439

B success rateiterations

90 55 100 100

9998 13251 5165 4443

(5) A success rateunique comb

90 59 97 98

9328 10027 5077 4636

B success rateiterations

90 56 99 99

10047 13327 6569 5624

(8) A success rateunique comb

91 59 97 98

9595 9712 5788 5348

B success rateiterations

90 56 99 99

10152 13410 7350 6686

Table 7 Performance for optimizing the combination of six drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f6a(x) (2) A success rateunique comb

91 43 100 100

12800 12141 4768 4326

B success rateiterations

90 43 100 100

13526 16626 5314 5036

(5) A success rateunique comb

90 44 99 99

12623 12472 6210 5983

B success rateiterations

90 45 100 100

13968 16759 7637 7361

(8) A success rateunique comb

90 46 98 98

12047 13024 7232 6988

B success rateiterations

90 46 98 98

14122 16819 8752 8343

f6b(x) (2) A success rateunique comb

91 43 100 100

5099 9710 3414 2377

B success rateiterations

94 44 100 100

12407 16460 4365 2935

(5) A success rateunique comb

90 42 100 100

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 13 of 15

ARU algorithm continued to substantially outperformother stochastic search algorithms [911] demonstratingthat it is better suited for practical drug optimizationapplicationsOne interesting observation is that the performance of

the Gur Game algorithm is typically not very sensitiveto measurement noise In fact in some cases its perfor-mance even improves as the noise level goes up Themain reason for this phenomenon is as follows As dis-cussed earlier the Gur Game algorithm does not adaptto the observed drug response function and for this rea-son its overall performance crucially depends onwhether or not its predetermined FSA matches the drugresponse function at hand As a result if the FSA doesnot match the original drug response function well iro-nically enough the measurement noise may perturb thesearch process in such a way that improves the overallperformance In this sense the fact that the Gur Gamealgorithm is not very sensitive to measurement noisereflects its inaptitude for handling various types of drugresponse functions rather than its robustness to randomfluctuations and noise in the measured drug response

ConclusionsIn this paper we proposed a novel stochastic searchalgorithm called the adaptive reference update (ARU)algorithm which can be effectively used for optimizingthe composition of combinatory drugs The proposedalgorithm intelligently utilizes the drug response valuesobserved in the past to reliably predict how to benefi-cially update the drug concentrations to improve thedrug response As we demonstrated throughout thispaper the proposed algorithm addresses several short-comings of previous drug optimization algorithms[911] thereby improving the overall search perfor-mance Numerical experiments based on various typesof multi-drug response functions show that the ARUalgorithm results in a higher success rate (ie higherprobability of identifying a potent drug combination)while requiring significantly fewer drug response evalua-tions Furthermore the proposed algorithm is robust to

random measurement noise where its search perfor-mance is not substantially affected in the presence ofnoise and degrades gracefully as the noise levelincreases Throughout our experiments we used a = 1for the ARU algorithm as well as the previous searchalgorithm [11] As discussed earlier the parameter acontrols the randomness of the search by determininghow much weight we should give to the drug responsevalues that we observed in the past In general unlessthe observations are very noisy or the underlying drugresponse function is assumed to be extremely nonlinearit would be best to set the parameter to the largestallowed value (ie a = 1) so that we fully utilize thepast observations for making our best informed guessabout the beneficial drug update strategy For compari-son we also repeated our simulations using a = 05 anda = 075 whose results are summarized in Table S1 -Table S7 (see Additional file 1) We can see from theseresults that a = 1 indeed leads to the best performancefor the drug response functions and the noise levels wehave considered in this paper

Additional material

Additional file 1 Performance of the ARU algorithm Furtherperformance evaluation results of the proposed Adaptive ReferenceUpdate (ARU) algorithm

AcknowledgementsBased on ldquoEfficient combinatorial drug optimization through stochasticsearchrdquo by Mansuck Kim and Byung-Jun Yoon which appeared in GenomicSignal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop oncopy 2011 IEEE [19]This work was supported in part by the National Science Foundationthrough NSF Award CCF-1149544This article has been published as part of BMC Genomics Volume 13Supplement 6 2012 Selected articles from the IEEE International Workshopon Genomic Signal Processing and Statistics (GENSIPS) 2011 The fullcontents of the supplement are available online at httpwwwbiomedcentralcombmcgenomicssupplements13S6

Authorsrsquo contributionsConceived and developed the algorithm MK BJY Performed theexperiments MK Analyzed the data and wrote the paper MK BJY

Table 7 Performance for optimizing the combination of six drugs in the presence of noise (Continued)

4738 9707 3499 2798

B success rateiterations

94 44 100 100

12786 17049 4808 3246

(8) A success rateunique comb

89 42 100 100

4544 9699 4574 3532

B success rateiterations

94 44 100 100

13331 17755 5453 3915

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 14 of 15

Table 6 Performance for optimizing the combination of five drugs in the presence of noise (Continued)

(8) A success rateunique comb

10 13 100 100

71 202 1445 1314

B success rateiterations

9 13 100 100

1914 708 1823 1565

f5b(x) (2) A success rateunique comb

89 55 100 100

9179 10263 4071 3439

B success rateiterations

90 55 100 100

9998 13251 5165 4443

(5) A success rateunique comb

90 59 97 98

9328 10027 5077 4636

B success rateiterations

90 56 99 99

10047 13327 6569 5624

(8) A success rateunique comb

91 59 97 98

9595 9712 5788 5348

B success rateiterations

90 56 99 99

10152 13410 7350 6686

Table 7 Performance for optimizing the combination of six drugs in the presence of noise

Noiselevel

Searchtype

Performancemetric

Gur Game(simultaneous)

Gur Game(sequential)

Previous search algorithm[11](a = 1)

ARU algorithm(proposed) (a = 1)

f6a(x) (2) A success rateunique comb

91 43 100 100

12800 12141 4768 4326

B success rateiterations

90 43 100 100

13526 16626 5314 5036

(5) A success rateunique comb

90 44 99 99

12623 12472 6210 5983

B success rateiterations

90 45 100 100

13968 16759 7637 7361

(8) A success rateunique comb

90 46 98 98

12047 13024 7232 6988

B success rateiterations

90 46 98 98

14122 16819 8752 8343

f6b(x) (2) A success rateunique comb

91 43 100 100

5099 9710 3414 2377

B success rateiterations

94 44 100 100

12407 16460 4365 2935

(5) A success rateunique comb

90 42 100 100

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 13 of 15

ARU algorithm continued to substantially outperformother stochastic search algorithms [911] demonstratingthat it is better suited for practical drug optimizationapplicationsOne interesting observation is that the performance of

the Gur Game algorithm is typically not very sensitiveto measurement noise In fact in some cases its perfor-mance even improves as the noise level goes up Themain reason for this phenomenon is as follows As dis-cussed earlier the Gur Game algorithm does not adaptto the observed drug response function and for this rea-son its overall performance crucially depends onwhether or not its predetermined FSA matches the drugresponse function at hand As a result if the FSA doesnot match the original drug response function well iro-nically enough the measurement noise may perturb thesearch process in such a way that improves the overallperformance In this sense the fact that the Gur Gamealgorithm is not very sensitive to measurement noisereflects its inaptitude for handling various types of drugresponse functions rather than its robustness to randomfluctuations and noise in the measured drug response

ConclusionsIn this paper we proposed a novel stochastic searchalgorithm called the adaptive reference update (ARU)algorithm which can be effectively used for optimizingthe composition of combinatory drugs The proposedalgorithm intelligently utilizes the drug response valuesobserved in the past to reliably predict how to benefi-cially update the drug concentrations to improve thedrug response As we demonstrated throughout thispaper the proposed algorithm addresses several short-comings of previous drug optimization algorithms[911] thereby improving the overall search perfor-mance Numerical experiments based on various typesof multi-drug response functions show that the ARUalgorithm results in a higher success rate (ie higherprobability of identifying a potent drug combination)while requiring significantly fewer drug response evalua-tions Furthermore the proposed algorithm is robust to

random measurement noise where its search perfor-mance is not substantially affected in the presence ofnoise and degrades gracefully as the noise levelincreases Throughout our experiments we used a = 1for the ARU algorithm as well as the previous searchalgorithm [11] As discussed earlier the parameter acontrols the randomness of the search by determininghow much weight we should give to the drug responsevalues that we observed in the past In general unlessthe observations are very noisy or the underlying drugresponse function is assumed to be extremely nonlinearit would be best to set the parameter to the largestallowed value (ie a = 1) so that we fully utilize thepast observations for making our best informed guessabout the beneficial drug update strategy For compari-son we also repeated our simulations using a = 05 anda = 075 whose results are summarized in Table S1 -Table S7 (see Additional file 1) We can see from theseresults that a = 1 indeed leads to the best performancefor the drug response functions and the noise levels wehave considered in this paper

Additional material

Additional file 1 Performance of the ARU algorithm Furtherperformance evaluation results of the proposed Adaptive ReferenceUpdate (ARU) algorithm

AcknowledgementsBased on ldquoEfficient combinatorial drug optimization through stochasticsearchrdquo by Mansuck Kim and Byung-Jun Yoon which appeared in GenomicSignal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop oncopy 2011 IEEE [19]This work was supported in part by the National Science Foundationthrough NSF Award CCF-1149544This article has been published as part of BMC Genomics Volume 13Supplement 6 2012 Selected articles from the IEEE International Workshopon Genomic Signal Processing and Statistics (GENSIPS) 2011 The fullcontents of the supplement are available online at httpwwwbiomedcentralcombmcgenomicssupplements13S6

Authorsrsquo contributionsConceived and developed the algorithm MK BJY Performed theexperiments MK Analyzed the data and wrote the paper MK BJY

Table 7 Performance for optimizing the combination of six drugs in the presence of noise (Continued)

4738 9707 3499 2798

B success rateiterations

94 44 100 100

12786 17049 4808 3246

(8) A success rateunique comb

89 42 100 100

4544 9699 4574 3532

B success rateiterations

94 44 100 100

13331 17755 5453 3915

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 14 of 15

ARU algorithm continued to substantially outperformother stochastic search algorithms [911] demonstratingthat it is better suited for practical drug optimizationapplicationsOne interesting observation is that the performance of

the Gur Game algorithm is typically not very sensitiveto measurement noise In fact in some cases its perfor-mance even improves as the noise level goes up Themain reason for this phenomenon is as follows As dis-cussed earlier the Gur Game algorithm does not adaptto the observed drug response function and for this rea-son its overall performance crucially depends onwhether or not its predetermined FSA matches the drugresponse function at hand As a result if the FSA doesnot match the original drug response function well iro-nically enough the measurement noise may perturb thesearch process in such a way that improves the overallperformance In this sense the fact that the Gur Gamealgorithm is not very sensitive to measurement noisereflects its inaptitude for handling various types of drugresponse functions rather than its robustness to randomfluctuations and noise in the measured drug response

ConclusionsIn this paper we proposed a novel stochastic searchalgorithm called the adaptive reference update (ARU)algorithm which can be effectively used for optimizingthe composition of combinatory drugs The proposedalgorithm intelligently utilizes the drug response valuesobserved in the past to reliably predict how to benefi-cially update the drug concentrations to improve thedrug response As we demonstrated throughout thispaper the proposed algorithm addresses several short-comings of previous drug optimization algorithms[911] thereby improving the overall search perfor-mance Numerical experiments based on various typesof multi-drug response functions show that the ARUalgorithm results in a higher success rate (ie higherprobability of identifying a potent drug combination)while requiring significantly fewer drug response evalua-tions Furthermore the proposed algorithm is robust to

random measurement noise where its search perfor-mance is not substantially affected in the presence ofnoise and degrades gracefully as the noise levelincreases Throughout our experiments we used a = 1for the ARU algorithm as well as the previous searchalgorithm [11] As discussed earlier the parameter acontrols the randomness of the search by determininghow much weight we should give to the drug responsevalues that we observed in the past In general unlessthe observations are very noisy or the underlying drugresponse function is assumed to be extremely nonlinearit would be best to set the parameter to the largestallowed value (ie a = 1) so that we fully utilize thepast observations for making our best informed guessabout the beneficial drug update strategy For compari-son we also repeated our simulations using a = 05 anda = 075 whose results are summarized in Table S1 -Table S7 (see Additional file 1) We can see from theseresults that a = 1 indeed leads to the best performancefor the drug response functions and the noise levels wehave considered in this paper

Additional material

Additional file 1 Performance of the ARU algorithm Furtherperformance evaluation results of the proposed Adaptive ReferenceUpdate (ARU) algorithm

AcknowledgementsBased on ldquoEfficient combinatorial drug optimization through stochasticsearchrdquo by Mansuck Kim and Byung-Jun Yoon which appeared in GenomicSignal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop oncopy 2011 IEEE [19]This work was supported in part by the National Science Foundationthrough NSF Award CCF-1149544This article has been published as part of BMC Genomics Volume 13Supplement 6 2012 Selected articles from the IEEE International Workshopon Genomic Signal Processing and Statistics (GENSIPS) 2011 The fullcontents of the supplement are available online at httpwwwbiomedcentralcombmcgenomicssupplements13S6

Authorsrsquo contributionsConceived and developed the algorithm MK BJY Performed theexperiments MK Analyzed the data and wrote the paper MK BJY

Table 7 Performance for optimizing the combination of six drugs in the presence of noise (Continued)

4738 9707 3499 2798

B success rateiterations

94 44 100 100

12786 17049 4808 3246

(8) A success rateunique comb

89 42 100 100

4544 9699 4574 3532

B success rateiterations

94 44 100 100

13331 17755 5453 3915

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 14 of 15

Competing interestsThe authors declare that they have no competing interests

Published 26 October 2012

References1 Borisy AA Elliott PJ Hurst NW Lee MS Lehar J Price ER Serbedzija G

Zimmermann GR Foley MA Stockwell BR Keith CT Systematic discoveryof multicomponent therapeutics Proc Natl Acad Sci USA 20031007977-7982

2 Chesney MA Ickovics J Hecht FM Sikipa G Rabkin J Adherence anecessity for successful HIV combination therapy AIDS 1999 13(Suppl A)S271-278

3 De Francesco R Migliaccio G Challenges and successes in developingnew therapies for hepatitis C Nature 2005 436953-960

4 Fitzgerald JB Schoeberl B Nielsen UB Sorger PK Systems biology andcombination therapy in the quest for clinical efficacy Nat Chem Biol2006 2458-466

5 Sawyers CL Cancer mixing cocktails Nature 2007 449993-9966 Zimmermann GR Lehar J Keith CT Multi-target therapeutics when the

whole is greater than the sum of the parts Drug Discov Today 20071234-42

7 Calzolari D Bruschi S Coquin L Schofield J Feala JD Reed JCMcCulloch AD Paternostro G Search algorithms as a framework for theoptimization of drug combinations PLoS Comput Biol 2008 4e1000249

8 Feala JD Cortes J Duxbury PM Piermarocchi C McCulloch ADPaternostro G Systems approaches and algorithms for discovery ofcombinatorial therapies Wiley Interdiscip Rev Syst Biol Med 2010 2181-193

9 Wong PK Yu F Shahangian A Cheng G Sun R Ho CM Closed-loopcontrol of cellular functions using combinatory drugs guided by astochastic search algorithm Proc Natl Acad Sci USA 2008 1055105-5110

10 Wang Y Yu L Zhang L Qu H Cheng Y A novel methodology formulticomponent drug design and its application in optimizing thecombination of active components from Chinese medicinal formulaShenmai Chem Biol Drug Des 2010 75318-324

11 Yoon BJ Enhanced stochastic optimization algorithm for findingeffective multi-target therapeutics BMC Bioinformatics 2011 12(Suppl 1)S18

12 Zinner RG Barrett BL Popova E Damien P Volgin AY Gelovani JG Lotan RTran HT Pisano C Mills GB Mao L Hong WK Lippman SM Miller JHAlgorithmic guided screening of drug combinations of arbitrary size foractivity against cancer cells Mol Cancer Ther 2009 8521-532

13 Jelinek F Fast sequential decoding algorithm using a stack IBM Journalof Research and Development 1969 13(6)675-685

14 Viterbi A Omura J Principles of digital communication and coding McGraw-Hill Inc New York NY USA 1979

15 Tsetlin M Finite automata and modeling the simplest forms of behaviorUspekhi Matem Nauk 1963 81-26

16 Tung B Kleinrock L Using finite state automata to produce self-optimization and self-control Parallel and Distributed Systems IEEETransactions on 1996 7(4)439-448

17 Ji C Zhang J Dioszegi M Chiu S Rao E Derosier A Cammack N Brandt MSankuratri S CCR5 small-molecule antagonists and monoclonalantibodies exert potent synergistic antiviral effects by cobinding to thereceptor Mol Pharmacol 2007 7218-28

18 Storn R Price K Differential Evolution - A Simple and Efficient Heuristicfor global Optimization over Continuous Spaces Journal of GlobalOptimization 1997 11341-359[httpdxdoiorg101023A1008202821328][101023A1008202821328]

19 Kim M Yoon B Efficient combinatorial drug optimization throughstochastic search Genomic Signal Processing and Statistics (GENSIPS) 2011IEEE International Workshop on 4-6 December 2011 2011 33-35

doi1011861471-2164-13-S6-S12Cite this article as Kim and Yoon Adaptive reference update (ARU)algorithm A stochastic search algorithm for efficient optimization ofmulti-drug cocktails BMC Genomics 2012 13(Suppl 6)S12

Submit your next manuscript to BioMed Centraland take full advantage of

bull Convenient online submission

bull Thorough peer review

bull No space constraints or color figure charges

bull Immediate publication on acceptance

bull Inclusion in PubMed CAS Scopus and Google Scholar

bull Research which is freely available for redistribution

Submit your manuscript at wwwbiomedcentralcomsubmit

Kim and Yoon BMC Genomics 2012 13(Suppl 6)S12httpwwwbiomedcentralcom1471-216413S6S12

Page 15 of 15