15
1,2 1,2 3 1 4 1 {agriesma,bka,rschlatte}@iist.unu.edu 2 [email protected] 3 [email protected] 4 [email protected]

{agriesma,bka,rschlatte}@iist.unu.edu [email protected] ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

Testing Con urrent Obje ts withAppli ation-Spe i� S hedulers⋆Rudolf S hlatte 1,2, Bernhard Ai hernig 1,2, Frank de Boer 3,Andreas Griesmayer 1, and Einar Bro h Johnsen 4

1 International Institute for Software Te hnology, United Nations University(UNU-IIST), Ma ao S.A.R., China{agriesma,bka,rschlatte}@iist.unu.edu

2 Institute for Software Te hnology, Graz University of Te hnology, [email protected]

3 CWI, Amsterdam, [email protected]

4 Department of Informati s, University of Oslo, [email protected] t. In this paper, we propose a novel approa h to testing exe- utable models of on urrent obje ts under appli ation-spe i� s hedul-ing regimes. Method a tivations in on urrent obje ts are modeled as a omposition of symboli automata; this omposition expresses all pos-sible interleavings of a tions. S heduler spe i� ations, also modeled asautomata, are used to onstrain the system exe ution. Test purposes areexpressed as assertions on sele ted states of the system, and weakestpre ondition al ulation is used to derive the test ases from these testpurposes. Our new testing te hnique is based on the assumption that wehave full ontrol over the (appli ation-spe i� ) s heduler, whi h is the ase in our exe utable models under test. Hen e, the enfor ed s hedulingpoli y be omes an integral part of a test ase. This ta kles the problemof testing non-deterministi behavior due to s heduling.1 Introdu tionIn this paper we address the problem of testing exe utable high-level behavioralmodels of on urrent obje ts. In ontrast to multi-threaded exe ution models forobje t-oriented programs su h as, e.g., the Java model for the parallel exe utionof threads, we onsider in this paper a model of obje t-oriented omputationwhi h des ribes a method all in terms of the generation of a orrespondingpro ess in the allee. The on urrent exe ution of obje ts then naturally arisesfrom asyn hronous method alls, whi h do not suspend while waiting for thereturn value from the method alls. Obje ts exe ute their internal (en apsulated)pro esses in parallel. In this setting, the s heduling of the internal pro esses of

⋆ This resear h was arried out as part of the EU FP6 proje t Credo: Modeling andanalysis of evolutionary stru tures for distributed servi es (IST-33826).

Page 2: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

an obje t dire tly a�e ts its behavior (both its fun tional and non-fun tionalbehavior). Therefore, a ru ial aspe t of the analysis of on urrent obje ts isthe analysis of the intra-obje t s heduling of pro esses. In ontrast to s hedulingon the operating-system level, the obje t-level s heduling poli ies will be �ne-tuned a ording to the appli ation requirements. We all this appli ation-spe i� s heduling. In this paper we introdu e a novel testing te hnique for on urrentobje ts under appli ation-spe i� s heduling regimes.We develop a testing te hnique for on urrent obje ts in the ontext ofCreol [9, 4℄, a high-level modeling language whi h allows for the abstra tion fromimplementation details related to deployment, distribution, and data types. Thesemanti s of this language is formalized in rewriting logi [11℄ and exe utes onthe Maude platform [3℄. As su h the Creol modeling language also allows for thesimulation, testing, and veri� ation of properties of on urrent obje t models,based on exe ution on the Maude platform as des ribed by formal spe i� ations.One of the main ontributions of this paper is a formal testing te hnique for thislanguage whi h integrates formal spe i� ations of appli ation-spe i� s hedulingregimes at an abstra tion level whi h is at least as high as that of the modelinglanguage. The novelty of this approa h is that it takes the s heduling poli y asan integral part of a test ase in order to ontrol its exe ution.In order to spe ify test ases in our formal testing te hnique, we �rst de-velop suitable behavioral abstra tions of the me hanisms for syn hronizing thepro esses within an obje t, as featured by the modeling language. The integra-tion of these behavioral abstra tions and the formal spe i� ation of a parti ulars heduling regime provides the formal basis for the generation of test ases. Forthe formal spe i� ation of test purposes we use assertions whi h express requiredproperties of the obje t state (or a suitable abstra tion thereof). Test ases arethen generated by applying a weakest pre ondition al ulus in order to �nd anabstra t behavior whi h satis�es the assertions [8℄. The exe ution of a test aseon the Maude platform requires instrumenting the Maude interpreter of Creol'soperational semanti s su h that it will enfor e the embodied s heduling poli yon the pro esses of the parti ular on urrent obje t whi h is onsidered by thetest ase. Parti ular test ases address the behavior of the on urrent obje tmodel under a given, formally de�ned s heduling regime. If su h a test ase failsto rea h its goal (test purpose), this might indi ate a problem with the givens heduling poli y. Hen e, the relevan e of this ontribution for modeling obje t-oriented systems in general is that it also allows the spe i� ation and analysis ofs heduling issues in an early stage of design, as an integral part of the high-levelmodels. However, in the following dis ussion we fo us on the important aspe tof ontrolling test- ase exe ution by enfor ing a s heduling regime.Paper overview. The rest of this paper is organized as follows: Se tion 2introdu es the Creol language and exe utable modeling. Se tion 3 gives a high-level overview and s ope for our approa h to testing. Se tion 4 explains themodel-ling approa h used, in luding the high-level spe i� ation of s hedulingpoli ies. Se tion 5 dis usses the details of test ase generation and exe ution.Finally, Se tion 6 dis usses related work and Se tion 7 on ludes the paper.

Page 3: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

sr::= s | s;return e L ::= class C(v) {T f ; M}

v ::= f | x M::= T m (T x) {T x; sr}b ::= true | false | v e ::= v | new C(v) | e.get | e!m(e) | null | this | callerT::= C | Bool | Void s ::= v := e | await g | skip | s; sg ::= b | v? | g ∧ g | if g then s fi | releaseFig. 1. The language syntax. Variables v are �elds (f) or lo al variables (x), and C isa lass name.2 Creol and Exe utable ModelingIn the design of omponent-based or obje t-oriented systems, it may be desirableto introdu e a separation of on erns between business ode, dealing with thefun tionality of the software unit, and syn hronization ode, dealing with thelo al s heduling of di�erent omputing a tivities. Creol is a high-level exe utablemodeling language for on urrent obje ts in whi h su h s heduling may be leftunderspe i�ed [9℄. The language has a formal semanti s de�ned in rewritinglogi [11℄ and exe utes on the Maude platform [3℄. This allows various analysiste hniques to be developed and applied to the Creol models, in luding, e.g.,pseudo-random simulation and breadth-�rst sear h through the exe ution spa e.In ontrast to, e.g., Java, ea h Creol obje t en apsulates its state; i.e., allexternal manipulation of the obje t state happens through alls to the obje t'smethods. Ea h pro ess orresponds to the a tivation of one of the obje t's meth-ods. In addition, obje ts exe ute on urrently: ea h obje t has a pro essor ded-i ated to exe uting the pro esses of that obje t, so pro esses in di�erent obje tsexe ute in parallel. In Creol, method alls are asyn hronous and assigned to so- alled futures [4℄. Only one pro ess may be a tive in an obje t at a time; theother pro esses in the obje t are suspended. We distinguish between blo king apro ess and releasing a pro ess. Blo king auses the exe ution of the pro ess tostop, but does not let a suspended pro ess resume. Releasing a pro ess suspendsthe exe ution of that pro ess and lets another (suspended) pro ess resume. Thus,if a pro ess is blo ked there is no exe ution in the obje t, whereas if a pro ess isreleased another pro ess in the obje t may exe ute. Although pro esses need notterminate, the exe ution of several pro esses within an obje t may be ombinedusing release points within method bodies. Release points may in lude pollingoperations on futures, to he k for the arrival of replies to asyn hronous method alls. At a release point, the a tive pro ess may be released and some suspendedpro ess may resume.Syntax. The language syntax of the subset of Creol used in this paper is pre-sented in a Java-like style in Fig. 1. For the purpose of this paper, we emphasizethe di�eren es with Java and fo us on the spe i� ation of a single lass. Atpresent, we omit some features of Creol, in luding inheritan e and method alls.Expressions e are standard apart from the asyn hronous method all e!m(e), the(blo king) read operation v.get, and the pseudo-variable caller whi h refers

Page 4: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

class batch_queue(Nat x) {Nat wc, batch, comein // waiting clients, barrier sizeSeq[Object] display // queue of registered client objects

Void batch_queue() { batch := x; wc := 0; comein := 0 }

Void register() {wc := wc+1;if wc ≥ batch then comein := batch fi;await comein > 0;comein := comein - 1;wc := wc-1;display := (display;caller);

}} Fig. 2. Motivating example: The batch_queue lass.to the aller of the urrent method a tivation. Statements s are standard apartfrom release points await g and release. Guards g are onjun tions of Booleanexpressions b and polling operations v? on futures v. When the guard in an

await statement evaluates to false, the statement be omes a release, other-wise a skip. A release statement suspends the a tive pro ess and anothersuspended pro ess may be res heduled.Example. We onsider a version of barrier syn hronization given by the lassbatch_queue in Fig. 2.. In a batch_queue obje t, lients are pro essed inbat hes (of size batch, the parameter x to the onstru tor sets the size of thebat hes). A lient whi h registers must wait until enough lients have registeredbefore getting assigned a slot in the queue. For simpli ity, we represent the queueas a lo al variable display, whi h is a sequen e of lients (semi olon is the ap-pend operator on sequen es). Before any all to register will return, the obje twill ontain batch pro esses.When enough alls are waiting to be registered, thenext bat h of pro esses may pro eed by assigning the value of batch to display.It is easy to see that the order in whi h allers are added to the display sequen edepends on the internal s heduling of pro esses in the obje t.On e more, we mention that only a subset of Creol is presented in this paper;the interested reader is referred to e.g. [9℄.3 Testing and Testing MethodologyThe exe utable formal semanti s of the Creol language allows the appli ationof di�erent analysis te hniques. In this se tion we brie�y sket h our proposedmethodology for testing Creol appli ations on the Maude platform.

Page 5: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

Our methodology fo uses on testing run-time properties of Creol obje ts. Bythe very nature of Creol obje ts, of parti ular interest is to test run-time prop-erties of the obje t state under di�erent possible interleavings of its pro esses.In order to spe ify and exe ute su h tests we need an appropriate abstra tion ofpro esses whi h fo uses on their interleavings as des ribed by the ontrol stru -ture of their release points. We do so by modeling the internal �ow of ontrolwithin a pro ess between its release points into atomi blo ks onsisting of se-quen es of assignments. The release points of a pro ess themselves then an berepresented by the states of a �nite automaton, also alled a method automaton(be ause pro esses are generated by method alls). The transitions of a methodautomaton involve the assignments and a guard on the obje t state whi h spe -i�es the enabling ondition of the orresponding atomi blo k. We assume givena �nite set of internal pro esses in an obje t, re�e ting the message queue ofin oming method alls for the obje t. The possible interleavings of this initiallygiven �nite set of pro esses is thus abstra ted into the interleavings of theirautomata representations.S heduler automata further onstrain the possible interleavings by means ofabstra t representations of the enabling onditions of the method automata. Theautomati ally generated s heduled system automaton representing the possibleinterleavings of the method automata and the s heduler automaton is instru-mented with test purposes, expressed as Boolean onditions over the methodautomata's state variables, that are atta hed to states.5To ompute test ases for a test purpose we sear h for paths that rea h andful�ll the test purpose. We generate a set of su h test ases by omputing a test�harness� des ribing all paths in the model that will rea h the test purpose. Tothis end, we use weakest pre ondition omputation to propagate the onditionsto the initial state of the system. The ondition at the initial state des ribes thevalues that state variables an take for exe uting that test ase, re�e ting thea tual parameters to the method alls in the message queue. Ea h possible paththat rea hes the ondition(s) is its own test ase.The exe ution of a test on the Maude platform then he ks whether theparti ular interleaving of the method automata des ribed by the path in thesystem automaton an be realized by the Maude implementation of the Creolobje t su h that it satis�es the onditions.4 Combining Method Automata and S heduling Poli iesIn this se tion, we present the symboli transition system onstru tion used tospe ify the system's behavior. We adapt the symboli transition systems of [13℄,using shared variables for ommuni ation instead of input/output a tions.Syntax A Symboli Transition System is a tuple 〈Q, q0, T, V 〉, where:5 Computing test ases that rea h a ertain ondition in the program an be donewith onditions that are simply true .

Page 6: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

wc := wc + 1comein := (wc >= batch) ? batch : comein

comein := comein - 1wc := wc - 1display := (display ; caller)

a b c[ comein > 0 ]

Fig. 3. Method Automaton of the register() method� Q is a �nite set of lo ations qi, i ≥ 0� q0 ∈ Q is the initial lo ation� V is a set of variables� T is a set of transitions of the form 〈q, g, S, q′〉, where• q ∈ Q is the sour e lo ation• g is a Boolean guard expression over V

• S is a sequen e of assignment statements hanging the value of somev ∈ V

• q′ ∈ Q is the target lo ationSemanti s A state is a pair 〈q, v〉 onsisting of a lo ation q and a valuation vfor the variables. For the initial state, q = q0. Let eval be the fun tion mappingan expression and a valuation to a result6. Then, for a state 〈q, v〉, exe uting atransition 〈q, g, S, q′〉 results in a new state 〈q′, v′〉 where the new valuation v′is the result of evaluating all assignment statements in S, using eval with theformer valuation v to al ulate new values for the a�e ted variables, providedthat eval(g, v) = true.4.1 Modeling Method Invo ations: Method AutomataInvo ations of methods on Creol obje ts are modeled by Method Automata, aslight extension of the symboli transition systems des ribed above.A method automaton is a tuple 〈m, Qm, qm0

, Tm, Vm,Valm〉 so that m is aunique identi�er, Q is a set of lo ations qmi et . Other than the systemati renaming of lo ations, the semanti s are the same as for symboli transitionsystems. Additionally, Valm is a mapping v ∈ Vm 7→ x giving initial values x toall variables v. (Con eptually, Valm models parameters passed to the method aswell as initial values of lo al variables.)A Creol method without release points is modeled as a method automatonwith only beginning and end state. Ea h release point is modeled as an interme-diate state where exe ution an swit h to another running method.By onvention, the names of the lo al variables in a method automaton arepre�xed with the unique identi�er m of the automaton, so that the names areunique in the presen e of multiple instan es of the automaton. This approa h6 In this paper, we use expressions over the integer and Boolean domains with theusual operations and semanti s.

Page 7: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

is su� ient sin e ea h invo ation of a Creol method is modeled by its own au-tomaton. Names of instan e variables, su h as wc and display in Fig. 3 arenot pre�xed in this way, sin e every method automaton has a ess to the sameinstan e variables.4.2 Modeling Parallelism: The System AutomatonA on�guration of multiple method invo ations running in parallel is modeledas a symboli transition system as well. We shall refer to su h an automaton asa system automaton.De�nition 1. Let Ai = 〈mi, Qmi, qmi

0, Tmi

, Vmi,Valmi

〉 be method automata(for 1 ≤ i ≤ n). De�ne the omposition of A1, . . . , An as a system automatonA = 〈Q, q0, T, V,Val〉 su h that

Q = {〈mi, qm1 , . . . , qmn〉 | ∀0 < j ≤ n : qmj ∈ Qmj

}qo = 〈m1, q

m1

0, . . . , qmn

0〉

T =

〈q, g, S, q′〉

q = 〈ml, qm1 , . . . , qmi , . . . , qmn〉 ∧

q′ = 〈mi, q′m1 , . . . , q′mi , . . . , q′mn〉 ∧

〈qmi , g, S, q′mi〉 ∈ Tmi∧ ∀j 6= i : q′mj = qmj

V =⋃

0<i≤n Vmi

Val =⋃

0<i≤n ValmiThe semanti s of exe uting a transition of the system automaton is that ofexe uting the transition of one of the parti ipating method automata (qmi;

q′mi), leaving the state of all other method automata invariant (q′mj = qmj ).Further note that the �rst element of the system automaton's state designates themethod automaton whi h did the previous transition (for the initial state, it isarbitrarily set to m1). Be ause of this, the transitions of the system automaton an be attributed ba k to a parti ular method automaton; this will be omeimportant in s heduling.4.3 Modeling S hedulers: The S heduler AutomataThe system automaton as de�ned in Se tion 4.2 does not pla e restri tions onwhi h method automaton exe utes at ea h step beyond the guards of the methodautomata transition themselves. We use a s heduler automaton to express addi-tional restri tions on method automata exe ution in the system automaton.A s heduler automaton is modeled as a labeled transition system. It is usedto strengthen the guards on the transitions of a system automaton omposed ofmethod automata m1 . . . mn, and hen e, restri t whi h method(s) are allowed torun.De�nition 2. Let A be a system automaton for methods m1, . . . , mn. De�ne as heduler for A as an automaton S = 〈Q, q0, T 〉 su h thatQ = {mi | 1 ≤ i ≤ n}q0 = m1

T = {〈q, g, q′〉 | q ∈ Q ∧ q′ ∈ Q ∧ g ∈ G(A)}

Page 8: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

m1 m2

ready(m1)

ready(m1)

¬ready(m1)

¬ready(m1)

m1 m2

¬ready(m2)

ready(m1)

ready(m2)

¬ready(m1)

Fig. 4. Example s heduler automata: priority (left), round-robin (right)The transitions on a s heduler automaton have guards g ∈ G(A) in theform of readiness predi ates that are de�ned in the following way: Given asystem automaton A for methods m1, . . . , mn, G(A) is de�ned indu tively byready(mi) ∈ G(A) and ¬ready(mi) ∈ G(A) for 1 ≤ i ≤ n, and g1 ∧ g2 ∈ G(A)and g1 ∨ g2 ∈ G(A) if g1, g2 ∈ G(A). The expression ready(mi) denotes a predi- ate whi h is true whenever the method automaton mi has at least one enabledtransition (i.e., whose guard evaluates to true) in the urrent state of A.The s heduler automaton has n states, one for ea h method automaton in thesystem automaton. Ea h s heduler state is labeled with one method automaton'sunique identi�er mi. The label on the urrent state of the s heduler automatonnames the method automaton that exe uted the most re ent transition of thesystem automaton. By de�nition, m1 is the s heduler automaton's initial state.Figure 4 shows two s heduling automata, both for a system automaton withtwo method automata m1 and m2: a simple priority s heduler that always givespreferen e to m1 over m2, and a round-robin s heduler.4.4 Integration of the S heduler and the System AutomatonThe s heduling of tasks in a system automaton a ording to the poli y expressedby a spe i� s heduler automaton is done in the following way:For ea h state q = 〈mk, . . .〉 of the system automaton, �nd the orrespondingstate mk of the s heduler automaton. For ea h transition t = 〈q, g, S, q1〉 in thesystem automaton, take the s heduler automaton's transition that enables t, i.e.the transition that leads to the s heduler state mi if q1 = 〈mi, . . .〉. If there isno su h s heduler transition, remove the transition from the system automaton(sin e the s heduler does not allow the method automaton mi to run after mk).Otherwise, strengthen the guard on the transition t by the guard expression onthe s heduler transition from mk and mi, repla ing all sub-expressions ready(mx)with the disjun tion of the guards on all transitions of method automaton mxin its urrent state.We refer to a system automaton whi h is s heduled by a s heduler automatonas a s heduled system automaton. Formally, we de�ne the expansion of readinesspredi ates for spe i� states of a system automaton and a s heduled systemautomaton as follows.De�nition 3. Let A = 〈Q, q0, T, V,Val〉 be a system automaton for the methodsm1, . . . , mn. For a state q ∈ Q and a s heduler guard g ∈ G(A), s heduler guard

Page 9: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

[ x = 5 ]

[ x = 5 ]

[ ¬(x = 5) ]

x := 5

[ ¬(x = 5) ]

x := 5

[ ¬(x = 5) ]

x := 5

[ ¬(x = 5) ]

x := 5

[ x = 5 ]

[ x = 5 ]

A

B

[ x = 5 ]

c

d

x := 5

1,A,c 1,A,d

2,A,c 2,A,d

1,B,c 1,B,d

2,B,c 2,B,dFig. 5. Two simple method automata and a system automaton onsisting of the twoautomata running in parallel under the priority s heduler of Figure 4 (guards in boldadded by the s heduler).expansion is a fun tion [[g]]q, indu tively de�ned as follows:[[ready(mi)]]q =

{g | 〈q, g, S, q1〉 ∈ T ∧ q1 = 〈mi, qm1 , . . . , qmn〉}

[[¬ready(mi)]]q = ¬[[ready(mi)]]q[[g1 ∨ g2]]q = [[g1]]q ∨ [[g2]]q[[g1 ∧ g2]]q = [[g1]]q ∧ [[g2]]qIn the �rst part of De�nition 3, we use the disjun tion on a set to denote thedisjun tion of all the elements in the set.De�nition 4. Let A = 〈QA, qA

0 , TA, VAValA〉 be a system automaton for meth-ods m1, . . . , mn and let S = 〈QS , qS0, TS〉 be a s heduler. De�ne a s heduledsystem as an automaton SA = 〈Q, q0, T, V,Val〉 su h that

Q = QA

q0 = qA0

T =

{

〈q, g, S, q′〉

q = 〈ml, qm1 , . . . , qmn〉 ∧ q′ = 〈mi, q

′m1 , . . . , q′mn〉∧〈q, g′, S, q′〉 ∈ TA ∧ 〈ml, g

′′, mi〉 ∈ TS ∧ g = (g′ ∧ [[g′′]]q)

}

V = VA

Val = ValAFor example, if the transition guard on the s heduler is [¬ready(m)] andautomaton m in its urrent state has two transitions with the guards [x <= 5]and [x > 5], then relevant guards on the transitions in the system automatonwill be strengthened with ¬(x <= 5 ∨ x > 5). Transitions whose guards redu eto false (as in this example) an be eliminated from the system automaton.5 Test Case Generation with WP and S hedulersWe use a s heduled system automaton SA (see De�nition 4) to test the Creolobje t it represents. SA ontains all runs an obje t an perform for a given

Page 10: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

reg1

[ false ]

reg2

[ comein>0 ]

reg2

[ comein>0 ]

reg2

[ ¬ comein>0 ]

reg1

reg1

[ comein>0 ]

reg2

reg1

[ comein > 0 ]

reg2 [ comein > 0 ]

reg2

reg1

[ false ]

reg1

reg2

reg2

[ comein>0 ! ¬ comein>0 ]

reg2

a,a m2,a,b m2,a,c

m1,b,a

m1,c,a

m2,b,b m1,b,b m2,b,c m1,b,c

m2,c,b m1,c,b m2,c,c m1,c,c

{ display = (caller1; caller2) }{ display = (caller1) } { display = (caller1) }{ display = (caller1) }

{ display = () } { display = () }

{ display = () }

reg1 ::= wc := wc + 1 comein := (wc >= 2) ? 2 : comein

reg2 ::= comein := comein - 1 wc := wc - 1 display := (display ; caller)Fig. 6. A s heduled system automaton with two method automata for the registermethod, under priority s heduling and with bat h size 2. Guard terms in bold are addedby the s heduler, states that are unrea hable under priority s heduling are dashed.initial message queue and s heduler. In the following, we give an approa h to omputing test ases of interest from this automaton.Spe i� ally, we de�ne how to ompute the weakest pre ondition (WP) fora s heduled system automaton and use this te hnique to generate test asesa ording to a test purpose.The intention of the test ases to generate is aptured by test purposes, whi hare abstra t spe i� ations of a tual test ases. In onforman e testing, the notionof a test purpose has been standardized [7℄:De�nition 5 (Test purpose, general). A des ription of a pre ise goal ofthe test ase, in terms of exer ising a parti ular exe ution path or verifying the omplian e with a spe i� requirement.In our setting, these requirements are expressed by assert statements in asystem automaton. The ondition p of an assert has to be ful�lled in all possibleruns leading to the assert. (For simpli ity, we will use p to refer to the assertionand its ondition synonymously.) To ompute test ases for a test purpose, wesear h for paths that rea h and omply with all its assert statements. Intuitively,this orresponds to omputing the weakest pre ondition for p. In the followingwe will, without loss of generality, on entrate on test purposes that an bespe i�ed with a single assertion. Conditions for the general ase are omputedby ombining the results from the single onditions.Figure 6 shows the graph of a system automaton that models two invo a-tions of the register method and batch size 2, s heduled with the prioritys heduler from Figure 4. This s heduler removes the edge from the initial state(a,a) to (m2,a,b) be ause both pro esses are enabled (with m1 having priority).

Page 11: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

Consequently, a portion of the state spa e of the system automaton be omesunrea hable in the s heduled system automaton and an be removed.Figure 6 also shows the additional onditions from s heduling on the edges.E.g., in state (m1,b,a) pro ess m2 is only enabled if comein is not > 0. The testpurpose is to ompute test ases to rea h state (m2, , ) with display = (caller1 ;caller2 ). We onstrain ourselves to only illustrate the WP omputation for thedisplay variable, whose omputed value is depi ted in urly bra kets. Computingthe WP to the initial state results in an empty display variable, for whi h allpaths rea h the desired state7. The a tual implementation must not blo k forthis input and must satisfy the assertion.To test the intermediate and �nal assertions on the Creol model, we reatea test harness H . The harness is onstru ted from the system automaton A asH = 〈QA, qA

0, TA, VA, c(QA)〉, with QA, qA

0, TA and VA re�e ting the systemautomaton, and c(QA) a ondition de�ned for ea h lo ation of A, representingthose valuations in a lo ation that only o ur in runs that eventually will rea hand omply with p. Thus, for every valuation in c(QA) two properties hold:(1) there is a transition su h that the destination is again in c(QA) and (fordeterminism) (2) there is no transition su h that the destination is not in c(QA).Using standard weakest-pre ondition predi ate transformers wp for our simplestatements S (assignments and sequential omposition only), we have:

cp(q) =∨

∀〈q,g,S,q′〉∈T

wp (S, c(q′)) ∧ g (1)c¬p(q) =

∀〈q,g,S,q′〉∈T

wp (S,¬c(q′)) ∧ g (2)c(q) = cp(q) ∧ ¬c¬p(q) (3)We ompute c(QA) iteratively by setting c0(q) = p for q = qp and c0(q) =

false for all other lo ations. The �rst iteration will result in all states that rea hp in one step, then those with distan e two and so forth. The iteration steps aresound: ea h iteration results in valuations that give valid test ases. This is animportant observation be ause although this pro ess always results in a �xedpoint for �nite state systems ( f. CTL model he king of AF p [2℄), the statespa e for STS is in�nite and the iteration might not terminate. Soundness allowsus to stop omputation after a ertain bound or amount of time even if no �xedpoint is rea hed yet. Any initial state in c(q0) gives valid test ases even if no�xed point an be omputed.The test ase for the s enario of Figure 6 onsists of the following:� A list of method invo ations (〈m1,register()〉, 〈m2,register()〉)� The priority s heduler from Figure 4� The initial value () for the instan e variable display� The test harness H , giving verdi ts at ea h s heduling de ision point7 The representation is strongly simpli�ed, exa t omputation will give more ondi-tions on the states and unveils that only the path using the edge (m1,b,a)(m2,b,b)is feasible.

Page 12: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

5.1 Test Case Exe utionThe test driver in Creol uses the s heduler to guide the Creol model and the testharness H to arrive at test verdi ts. The initial values and method parametersare hosen su h that ondition c(q0) is ful�lled, at ea h release point of the Creolobje t, the onditions on the harness are he ked. At ea h release point, thes heduler hooses among the enabled pro esses to ontinue the exe ution. Thereare two di�erent ways of arriving at a test verdi t of Fail :� If the Creol obje t does not ful�ll the urrent ondition of the harness, theimplementation of the last exe uted basi blo k violates the spe i� ation bythe method automaton.� If the ondition is ful�lled but no pro ess is enabled (the test pro ess dead-lo ks), the implementation fails to handle all the valuations that are requiredby the model.If the test harness arrives at the terminating state and the ondition is ful�lled,a test verdi t of Su ess is rea hed.Strengthening the Guards of the Harness. The omputation as shownabove uses the weakest pre ondition to rea h the test purpose p, or, in otherwords, the set of initial states that rea h the test purpose in every legal run.Input values that might miss p due to non-determinism are ignored. To a hieveoptimal test overage, however, it is desirable to sear h for all input values that an ful�ll the test purpose and add enough information to H for the test driverto guide the run to the desired state. In other words, instead of omputing thoseinitial states that will rea h p in every run, we want to ompute states for whi ha run exists.The annotated automaton provides us with a simple me hanism to a hievethis goal. For the ne essary adjustments we have a se ond look at the ompu-tation of c(QA). Formula (1) represents the states that an rea h p, while thosestates that an avoid p are removed using Formula (2). If we don't onsider c¬pin Formula (3), we ompute all valuations for whi h a run to p exists, but thetest driver has to perform the run on a trial an error basis: exe uting a statementand he king if the result still an rea h p, ba ktra king otherwise. To avoid thisoverhead, we add new guards g′ to H to restri t the runs to those valuationsthat always an rea h p:g′(< q, g, S, q′ >) = g ∧ wp(S, c(q′))Using g′ for the omputation of c(QA) results in all states for whi h a run to pexists, whi h easily an be seen by inserting g with g ∧ wp(S, c(q′)) in formulae(1) and (2):

c′p(q) =∨

∀〈q,g,S,q′〉∈T

wp (S, c(q′)) ∧ g ∧ wp (S, c(q′)) = cp(q)

Page 13: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

c′¬p(q) =∨

∀〈q,g,S,q′〉∈T

wp (S,¬c(q′)) ∧ g ∧ wp (S, c(q′)) = false

c′(q) = c′p(q)Using g′ as guards for the test driver ex ludes all transitions to states that annot rea h p. This allows to avoid unne essary ba ktra king while examiningall paths that an be extended to rea h the test purpose, resulting in a largervariety of possible runs and better overage. The approa h does not ome withoutobsta les though, g′ only points to states that an rea h p � the test driver needsto be able to dete t loops to make sure to �nally rea h it. Furthermore, a pathto p might not be available in the implementation. If the only available pathavoids p, the test driver has to ba ktra k to �nd a path to p.6 Related WorkWith the growing dependen y on distributed systems and the arrival of multi ore omputers, on urrent obje t-oriented programs form a resear h topi of in reas-ing importan e. Automata-based approa hes have previously been used to model on urrent obje t-oriented systems; for example, Kramer and Magee's FSP [10℄use automata to represent both threads and obje ts, abstra ting from spe i� syn hronization me hanisms. However, they do not address the issue of repre-senting spe i� s heduling poli ies that we onsider in this paper. S hönbornand Kyas [14℄ use Streett Automata to model fair s heduling poli ies of externalevents, with ontrolled s heduler suspension for on�gurations that deadlo k thes heduler.A lot of work is done in the area of s hedulability whi h mainly deals withthe question if a s heduler exists whi h is able to meet ertain timing onstraints(e.g., [12, 6℄), but does not look into the fun tional hanges imposed by di�er-ent appli ation-level s heduling poli ies. Established methods for testing obje t-oriented programs like unit-testing, on the other hand, deal with the fun tional-ity on a �ne grained level, but fail to he k for the e�e ts of di�erent s hedulers(see e.g., [18℄). Instead, the main hallenge for testing on urrent programs isto show that the properties of interest hold independent of the used s heduler.In ontrast, the approa h we have taken in this paper is to test properties of aprogram under a spe i� s heduling regime.Stone [15℄ was the �rst proposing the manipulation of the s hedules to iso-late failure auses in on urrent programs. Her idea was to redu e the non-determinism due to s heduling by inserting additional break points at whi h apro ess waits for an event of another pro ess. In Creol, this ould be a hieved byinserting additional await-statements. However, dealing with a modeling lan-guage, we prefer the more expli it restri tion of non-determinism by modelingthe s heduling poli y dire tly. More re ently, Edelstein et al. [5℄ manipulatedthe s heduler in order to gain higher test overage of on urrent Java programs.They randomly seeded sleep, yield or priority statements at sele ted pointsin order to alter the s heduling during testing. This approa h is based on the

Page 14: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

observation that a given s heduler behaves largely deterministi under onstantoperating onditions; by running existing tests under other s heduling strategies,additional timing-related errors are un overed. Choi and Zeller [1℄ hange s hed-ules of a program to show the ause of a problem for a failing test ase. They useDEJAVU, a apture/replay tool that re ords the thread s hedule and allows thereplay of a on urrent Java program in a deterministi way. Delta-debugging isused to systemati ally narrow down the di�eren e between a passing and failingthread s hedule. This approa h helps in order to he k if programs work underdi�erent s hedules, but unlike the method shown in this paper do not help inthe a tual generation of the test ase.Jasper et al. [8℄ use weakest pre ondition omputation to generate test asesespe ially tailored for a omplex overage riterion in single threaded ADA pro-grams. Rather than augmenting the model, they generate axioms des ribing theprogram and use a theorem prover to ompute its feasibility. More re ently, [17℄use weakest pre ondition to identify ause-e�e t hains in failing test ases tolo alize statements responsible for the error (fault lo alization). WP omputa-tion is furthermore used in several abstra tion algorithms to identify relevantpredi ates for removing infeasible paths in abstra t models. In [16℄, Tillmannand S hulte introdu e �parametrized unit tests�, whi h serve as spe i� ationsfor obje t oriented programs. They use symboli exe ution to generate the in-put values for the a tual test ases. However, none of these approa hes use WP omputation for test ase generation in on urrent systems.7 Con lusion and Future WorkThis paper presents an approa h to generating test ases for on urrent, obje t-oriented programs with appli ation-spe i� s hedulers. The s heduling poli ybe omes part of the test ase in order to ontrol its exe ution. We therefore in-trodu e an automaton approa h for spe ifying the behavior of both the systemand the s heduler, as well as its omposition and extension to a harness for a testdriver. Enfor ing a s heduling regime limits the non-deterministi interleavings ofbehavior, a well-known problem in testing and debugging of on urrent systems.A further important aspe t is that the separation of on erns between fun tion-ality and s heduling allows s heduling issues, whi h are ru ial in on urrentprograms, to be spe i�ed and tested at the abstra tion level of the exe utablemodeling language.In this paper, we expe t the method automata and s heduler to be givenas spe i� ations, and he k for omplian e with a given Creol implementation.A natural extension for future work is to automati ally onstru t the methodautomata from the Creol ode and he k against di�erent s hedulers for om-plian e. The test driver will be implemented within the Maude interpreter forCreol, whi h allows the test driver to in�uen e the s heduling.Further future work omprises the extension to s hedulers with internal stateto express more involved s heduling strategies and to extend our approa h withfurther features of obje t-oriented languages.

Page 15: {agriesma,bka,rschlatte}@iist.unu.edu aichernig@ist.tugraz ...projects.cwi.nl/credo/publications/testing... · pro cess resume. Releasing susp ends the execution of that pro cess

Referen es1. J.-D. Choi and A. Zeller. Isolating failure-indu ing thread s hedules. In Interna-tional Symposium on Software Testing and Analysis, pages 210�220. ACM, 2002.2. E. M. Clarke, O. Grumberg, and D. A. Peled. Model Che king. The MIT Press,Cambridge, Massa husetts, 1999.3. M. Clavel, F. Durán, S. Eker, P. Lin oln, N. Martí-Oliet, J. Meseguer, and J. F.Quesada. Maude: Spe i� ation and programming in rewriting logi . Theoreti alComputer S ien e, 285:187�243, Aug. 2002.4. F. S. de Boer, D. Clarke, and E. B. Johnsen. A omplete guide to the future. InR. de Ni ola, editor, Pro . 16th European Symposium on Programming (ESOP'07),volume 4421 of LNCS, pages 316�330. Springer, Mar. 2007.5. O. Edelstein, E. Far hi, Y. Nir, G. Ratzaby, and S. Ur. Multithreaded Java programtest generation. IBM Systems Journal, 41(1):111�125, February 2002.6. E. Fersman, P. Kr ál, P. Pettersson, and W. Yi. Task automata: S hedulability,de idability and unde idability. Information and Computation, 205(8):1149�1172,2007.7. ISO/IEC 9646-1: Information te hnology - OSI - Conforman e testing methodologyand framework - Part 1: General Con epts, 1994.8. R. Jasper, M. Brennan, K. Williamson, B. Currier, and D. Zimmerman. Test datageneration and feasible path analysis. In Pro eedings of the International sympo-sium on Software testing and analysis (ISSTA'94), pages 95�107. ACM, 1994.9. E. B. Johnsen and O. Owe. An asyn hronous ommuni ation model for distributed on urrent obje ts. Software and Systems Modeling, 6(1):35�58, Mar. 2007.10. J. Magee and J. Kramer. Con urren y: State Models & Java Programs. Wiley, 2edition, 2006.11. J. Meseguer. Conditional rewriting logi as a uni�ed model of on urren y. Theo-reti al Computer S ien e, 96:73�155, 1992.12. L. Nigro and F. Pupo. S hedulability analysis of real time a tor systems using oloured petri nets. In Con urrent Obje t-Oriented Programming and Petri Nets:Advan es in Petri Nets, volume 2001 of LNCS, pages 493�513. Springer, 2001.13. V. Rusu, L. du Bousquet, and T. Jéron. An approa h to symboli test generation.In Pro eedings of the 2nd International Conferen e on Integrated Formal Methods(IFM'00), volume 1945 of LNCS, pages 338�357. Springer, 2000.14. J. S hönborn and M. Kyas. A theory of bounded fair s heduling. In J. Fitzgeraldand A. Haxthausen, editors, International Colloquium on Theoreti al Aspe ts ofComputing (ICTAC), LNCS. Springer, 2008. (This volume).15. J. M. Stone. Debugging on urrent pro esses: A ase study. In Pro eedingsSIGPLAN Conferen e on Programming Language Design and Implementation(PLDI'88), pages 145�153. ACM, June 1988.16. N. Tillmann and W. S hulte. Parameterized unit tests. In Pro eedings of the 10thEuropean Software Engineering Conferen e / 13th ACM SIGSOFT Symposium onthe Foundations of Software Engineering (ESEC/FSE'05), pages 253�262. ACM,2005.17. C. Wang, Z. Yang, F. Ivan i , and A. Gupta. Whodunit? Causal analysis for ounterexamples. In International Symposium on Automated Te hnology for Ver-i� ation and Analysis (ATVA'06), volume 4218 of LNCS, pages 82�95. Springer,2006.18. E. J. Weyuker. Testing omponent-based software: A autionary tale. IEEE Soft-ware, pages 54�59, September 1998.