QoS Testing of Service-Based Applications

Maha Driss1,2, Yassine Jamoussi1 and Henda Hajjami Ben Ghezala1

1National School of Computer Sciences (ENSI)RIADI-GDL Laboratory

University of Manouba, Tunisiayassine.jamoussi@ensi.rnu.tnhenda.benghezala@ensi.rnu.tn2IRISA/INRIA, Rennes, France

maha.driss@irisa.fr

Abstract

Web services (WSs) are becoming increasingly popularbecause of their potential in several application domainsincluding e-Enterprise, e-Business, e-Government, ande-Science. Based on open XML standards, WS technologyallows the construction of massively distributed and looselycoupled applications. A service composition mechanismshould satisfy not only functional properties but alsonon-functional Quality of Service (QoS) ones. In thispaper, we introduce a discrete-events modeling approachfor Service-Based Applications (SBAs). This approach isoriented towards QoS evaluation through simulation.

Keywords: Service-Based Applications, Discrete-Events Simulation, QoS, Evaluation.

1. Introduction

In the past decades, the evolution of software systemsobserves continuous growth of the complexity levels andthe maturation of the networks and protocols. The cross-application integration and interoperability, the flexibilityof implementation and reconfiguration became essential re-quirements for the modern architectural styles. The mainchallenge is to simplify the integration of software appli-cations, regardless possible heterogeneity of implementingplatforms, protocols, and devices. Service-Oriented Ar-chitecture (SOA) utilizes services as the fundamental el-ements in order to deliver low-cost, flexible, and scalabledistributed applications [15] [22]. Web Services (WSs) areuniversally accessible software components advertised, dis-covered, and invoked over the Web [22]. The key aspect ofthis emerging paradigm is the use of standard technologies

such as: Web Services Description Language (WSDL) [10],Universal Description, Discovery, and Integration (UDDI)[4] and Simple Object Access Protocol (SOAP) [7]. Thesetechnologies aim to support the definition of services, theiradvertisement, and their binding for triggering purposes.The process-based composition of WSs is gaining a con-siderable momentum as an approach for the effective inte-gration of distributed, heterogeneous, and autonomous ap-plications [15]. Quality of service (QoS) is an importantconcern for Service-Based Applications (SBAs). QoS forWSs refers to various non-functional characteristics suchas response time, throughput, availability and security [6].Since many service providers provide similar services withcommon functionality, different WS processes can be com-posed satisfying the same user requirement. Users will dis-criminate these processes based on their QoS [28]. Also,WSs operate autonomously within a highly variable envi-ronment: the Web. As a result, their QoS may evolve rela-tively frequently to the dynamic and unpredictable nature ofthe execution environment (e.g., network resources, devicescharacteristics, etc.) [28]. In particular, during an executionof a composite service, the component services involvedmay change their QoS properties, others may become un-available, and still others may emerge. For the above rea-sons, the management and assurance of the quality aspectsof SBAs become of utmost importance.To achieve the desired quality of a SBA, different analyticalquality assurance techniques can be applied. The goal ofthese techniques is to evaluate QoS and to uncover faults inthe composed applications after their creation. One of therelevant techniques is testing. The goal of testing is to (sys-tematically) execute services or SBAs in order to uncoverfailures. During testing, the service or SBA which is testedis fed with concrete inputs and the produced outputs are ob-served [21] [18] [2]. The observed outputs can deviate from

the expected outputs with respect to functionality as well asQoS. When the observed outputs deviate from the expectedoutputs, a failure of the service or the SBA is uncovered.A special case of testing is simulation. The goal of the sim-ulation is to emulate the conversational behaviour of atomicWSs participating in the composition.In this paper, we adopt a discrete-events simulation to testWS compositions. We introduce a modeling approach forSBAs. This approach enables analytical description ofSBAs and allows QoS predictions in different status andconditions of the execution environment. We define a sim-ple quality model for WSs focusing on essential propertiesof QoS that play a critical role for the effective managementof WSs and that can be measured by simulation technique.The remainder of the paper is structured as follows: in Sec-tion 2, simulation issues are addressed. Experimental re-sults are documented in Section 3. Related research is dis-cussed in Section 4. This paper closes with concluding re-marks and future work.

2. Discrete-events simulation modeling of SBAs

Simulation is an important testing technique, as it allowsusers to tune and to evaluate software systems withoutexperiencing the cost of enacting them. In this work,we propose an SBA modeling approach that is orientedtowards performance evaluation through discrete-eventssimulation.Discrete-events simulation is a kind of qualitative descrip-tion of a dynamic system whose behaviour is event-driven.This technique is frequently used to analyze and predict theperformance of real word systems. Given the evolution ofthe operation of a system, we can analyze its behaviour andevaluate appropriate quality measures. [26] and [11] arefundamental works about discrete-event systems diagnosis.Discrete-events simulation is suitable to model the be-haviour of a SBA since it is composed of WSs which aredecentralized and dynamic. The interactions between WSscan be modeled by a synchronized composition of severallocal models.To elaborate our simulation model for SBAs, we are basedon the work presented in [20] that focuses on modeling dis-tributed applications. We model an SBA as a combinationof two types of entities: distributed application and networkinfrastructure entities. Our simulation model is shown inFigure 1.

2.1. Distributed application modeling

The operation of distributed application is based on theclient-server model. In this model, the client sends a setof requests to the server and the server sends a response

Figure 1. Simulation model for SBAs

back to the client for each request. The operation scenariois supported through specifying groups of actions:

• Processing: indicating data processing;

• Request: indicating invocation of a server process;

• Write: indicating data storage;

• Read: indicating data retrieval;

• Transfer: indicating data transfer between client andserver processes;

• Synchronize: indicating replica synchronization.

Each process is executed on a processing node. Processingaction indicates invocation of the processing unit of the cor-responding node and is characterized by the amount of datato be processed.Request action indicates invocation of a server process andis characterized by the name of the server, the name of theWS, its invoked interface and the required inputs. Requestaction implies activation of the network, since the requestand the reply must be transferred from the invoking to theinvoked process, and vice versa.There are two available actions for data storing, read andwrite, which are characterized respectively by the amountof the stored and retrieved data and the invoked server. Ob-servations and performance analysis of web services appli-cations have proven that SOAP messages are small and sim-ple [9].Transfer action is used to indicate SOAP messages ex-change between processes.Synchronize action is needed since replication of processesand data is a common practice in such distributed applica-tions. Synchronize action parameters include the process

replicas that must be synchronized and the amount of trans-ferred data.To describe the operation of a SBA, we proceed bytransforming the process behaviour written in BPEL intodiscrete-event actions. BPEL is a standard proposed byIBM and Microsoft along with several other companies tomodel composed Web services [1]. BPEL defines a gram-mar for describing the behaviour of a WS process. It iscomposed of fifteen activity types, some of them are basicactivities and the others are structured activities. Among thebasic activities, the most important ones are the following:

• The <receive> activity: is for accepting the triggeringmessage from another Web service;

• The <reply> activity: is for returning the response toits requestor;

• The <invoke> activity: is for invoking another Webservice.

The structured activities define the execution orders of theactivities inside their scopes. For example:

• The <sequence> activity: defines the sequential orderof the activities inside its scope;

• The <flow> activity: defines the concurrent relationsof the activities inside its scope.

Each activity can be translated into the discrete-event for-malism as one or several actions. Basic activities involveprocessing, request and data storing actions, while struc-tured ones involve transfer and synchronize actions.

2.2. Network infrastructure modeling

In the proposed modeling scheme, the network infras-tructure is considered as a collection of individual net-works and internetworks, exchanging messages through re-lay nodes (active communication devices e.g. routers andswitches). Communication element entity represents proto-col suites (i.e. routing protocols (OSI layers 2 and 3) andpeer-to-peer protocols (OSI layers 4-7)).According to the SOA, communication between processesis performed through exchanging SOAP messages.Figure 2 illustrates one way of making a remote call usingSOAP in OSI network reference model [5].First at application level, a native data object needs to beserialized into XML as SOAP request. Then, the SOAPmessage is passed to HTTP level. The HTTP layer, on theclient-side, needs to ’handshake’ with service-side by send-ing a ’POST’ request. This request initiated a TCP connec-tion. Once receiving ’HTTP: ACK’, the client-side HTTPbegins to send the whole SOAP message via TCP/IP. The

SOAP message may be partitioned into a set of small seg-ments at TCP layer. Appropriate headers and footers are at-tached to each segment as the segments are passed throughTransport, Network, Data Link layers, until reaching theNetwork Interface Card (NIC) at the physical layer. TheNIC is responsible for putting the packages onto the wire ata specific speed (network bandwidth) to next network de-vice (such as router or switch), till server NIC [5]. The pathfrom bottom (physical layer) to the top (application layer)on the service-side is opposite to the process on the client-side: the received packages are unpacked at each layer andforwarded to next layer for further retrieving.

Figure 2. Sending a SOAP request under OSI

2.3. Our simulation tool

We have conducted simulation experiments using NS-2simulator [13]. NS-2 is a discrete-events simulator; its codeis written in C++ with an OTcl interpreter as a front end.NS-2 is targeted at networking research. It provides sub-stantial support for simulation of TCP, routing, and mul-ticast protocols over wired and wireless networks. Themain advantages of such an object-oriented simulator arereusability and easy maintenance. To support SBA simula-tion, we have extended the C++ class hierarchy in order toimplement HTTP, SMTP and SOAP protocols.Our SBA Simulator (SBAS) is modular and includes: agraphical user interface, a BPEL generator, a simulationmodel generator, a models library and NS-2 simulator. Thearchitecture of SBAS is presented in Figure 3.User specifies the SBA under study using activity diagrams.SBAS constructs corresponding BPEL model. Simulationmodel is implemented as actions organized in the object hi-erarchy of the NS-2 simulator. When simulation has beencompleted, results are collected and subjected to outputQoS analysis.

Figure 3. SBAS architecture

3. Experimentation

As a running example, we use the Travel Planner (TP)composite service, whose BPEL code is sketched in Figure4, while Figure 5 illustrates the corresponding activitydiagram with its abstract services.

Figure 4. The TP BPEL specification

Simulation experiments were conducted in the testingplatform described in Figure 6. This platform consistsof two identical 1.86GHz CPUs machines connected via100Mbps LAN.

Figure 5. The TP activity diagram

Figure 6. Test platform

In this paper,we focus on evaluating only response time.Other QoS parameters (e.g., reliability, availability andscalability) are described and analyzed in our previousworks [12].Response Time corresponds to the total time needed bya WS to transform a set of inputs into outputs. ResponseTime (RT) for a service (s) can be computed as follows:

RT(s) = ST(s) + DT(s)

• Service Time (ST) is the time that the WS takes to per-form its task.

• Delay Time (DT) is the time taken to send/receiveSOAP messages.

DT(s) =TmsgTrans(s) + Tsynch(s) + Tapp(s)

– TmsgTrans represents the total cost to transfer aspecific amount of messages over network;

– Tsynch represents the overhead of the extra syn-chronization required by protocols;

– Tapp represents the time spent in business logicat application level.

For more details about TmsgTrans, Tsynch and Tapp mod-eling see the work described in [9].We have performed a set of simulation experiments, and wehave considered the average value. Our simulation resultsare summarized in Figure 7 and we conclude that service”BikeRental” is the fastest and the most scalable one amongall tested WSs. ”CarRental” service poses time problems soit must be changed.

Figure 7. Simulation results

4. Related work

To achieve the desired quality of an SBA, different ana-lytical quality assurance techniques can be employed. Thegoal of these techniques is to evaluate QoS and to uncoverfaults in the SBA after it has been created.The review of relevant literature in the field of quality as-surance for SBAs shows three major classes of analyticaltechniques and methods: (i) Testing, the goal of which isto (systematically) execute services or SBAs with prede-fined inputs in order to uncover failures, (ii) Monitoring,which observes services or SBAs as well as their contextduring their current execution, (iii) Static analysis, the aimof which is to systematically examine (without execution)an artifact (e.g., a service specification) in order to deter-mine certain properties or to ascertain that some predefinedproperties are met.These classes have been proposed in the software qualityassurance literature (e.g., see [21] and [18]) and have beenused in a recent overview of quality assurance approachesfor SBAs [2]. There have been numerous efforts by lead-ing research groups to use monitoring and static analysis toevaluate QoS and to discover problems in SBA execution.Relevant works in the area of SBAs monitoring are: [23][6] [3] [27], etc. Static analysis is adopted in [25] [14] [24],

etc.In this work, testing technique is used for analyzing theefficiency of SBAs. We adopt simulation approach to testSBAs. Simulation allows us to predict system performancein different status and load conditions of the execution envi-ronment. The predicted results are used to provide feedbackon the efficiency of the system. Simulating Web processesfor performance evaluation is a research area with little pre-vious work. Works in simulation that are the closest to oursare described by [19] [8] and [16].[19] proposes a model-theoretic semantics as well as dis-tributed operational semantics that can be used for simu-lation, validation, verification, automated composition andenactment of DAML-S-described SBAs. To provide a fullyservice description, Narayanan and McIlraith use the ma-chinery of situation calculus and its execution behaviour de-scribed with Petri Nets. They use the simulation and model-ing environment KarmaSIM to translate DAML-S markupsto situation calculus and Petri Nets. In this work, three veri-fication problems are simulated and analyzed: reachability,liveness and existence of deadlocks.Use of simulation for WSs is described also in [8]. Thiswork focused on problems related to SBA specification,evaluation and execution using Service Composition andExecution Tool (SCET). SCET allows to compose staticallya WS process with WSFL and to generate simulation modelthat can be processed by the JSIM simulation environment.In this work, Chandrasekaran et al. have enhanced WSFL toinclude QoS estimates obtained by performing simulationtests. Only the QoS response time dimension is supportedby the implementation of JSIM [8].In contrast to our approach, both [19] and [8] don’t considerexecution environment information in simulation model.[16] presents a framework whose aim is to support thedevelopment of self-optimizing, predictive and autonomicsystems for WS architectures. It adopts a simulation-basedmethodology, which allows predicting system QoS in dif-ferent status and load conditions.In contrast to [19] and [8], this work considers executionenvironment information in simulation model.This work focuses on simulating only atomic WSs. Alsoit proposes only one possible QoS optimization that is re-sponse time minimization. Enhancements are needed tosimulate SBAs and to add more optimization rules for QoSparameters.

5. Conclusion

We have presented a discrete-events simulation approachfor testing SBAs.We have contributed in the definition of a simulation modelfor SBAs. The proposed model is composed of two types ofentities: distributed application and network infrastructure.

Both are described in terms of their elementary components.Distributed application is described by a set of primitive ac-tions: Processing, Request, Write, Read, Transfer and Syn-chronize. Network infrastructure is considered as a collec-tion of individual networks and internetworks, exchangingmessages.Our modeling approach is oriented towards QoS evaluation.A simulation tool has been constructed for this purpose. Toshow the effectiveness of our approach, we have conducteda set of simulation experiments.One possible extension of our work is the support of dy-namic adaptations of SBAs. This requires specific testingand analysis techniques to verify the adaptation behaviour.

References

[1] Business process execution lan-guage for web services. Online:http://www.ibm.com/developerworks/library/specification/ws-bpel/, 2003.

[2] L. Baresi and E. DiNitto. Test and analysis of web services.Springer-Verlag GmbH, 2007.

[3] L. Baresi and S. Guinea. Towards dynamic monitor-ing of ws-bpel processes. Third International Conferenceof Service-Oriented Computing - ICSOC, pages 269–282,2005.

[4] T. Bellwood, S. Capell, L. Clement, J. Colgrave, M. J.Dovey, D. Feygin, A. H. R. Kochman, P. Macias,M. Novotny, M. Paolucci, C. von Riegen, T. Rogers,K. Sycara, P. Wenzel, and Z. Wu. Universal description,discovery and integration specification (uddi) 3.0.2. Online:http://uddi.org/pubs/uddi-v3.0.2-20041019.htm.

[5] D. Bertsekas and R. Gallager. Data Networks. Prentice Hall,1992.

[6] D. Bianculli and C. Ghezzi. Monitoring conversational webservices. IW-SOSWE ’07: 2nd international workshop onService oriented software engineering, pages 15–21, 2007.

[7] D. Box, D. Ehnebuske, G. Kakivaya, A. Layman,N. Mendelsohn, H. F. Nielsen, S. Thatte, and D. Winer.Simple object access protocol (soap) 1.1. Online:http://www.w3.org/TR/SOAP/, 2001.

[8] S. Chandrasekaran, J. A. Miller, G. A. Silver, I. B. Arpinar,and A. P. Sheth. Performance analysis and simulation ofcomposite web services. Electronic Markets, 13(2), 2003.

[9] S. Chen, B. Yan, J. Zic, R. Liu, and A. Ng. Evaluation andmodeling of web services performances. in ICWS ’06: Pro-ceedings of the IEEE International Conference on Web Ser-vices., 2006.

[10] R. Chinnici, J.-J. Moreau, A. Ryman, and S. Weerawarana.Web services description language (wsdl) version 2.0: Corelanguage. Online: http://www.w3.org/TR/wsdl20/.

[11] M. O. Cordier and S. Thi’ebaux. Event-based diagnosis forevolutive systems. In Proceedings of the Fifth Internationalworkshop on Principles of diagnosis(DX’94), pages 64–69,1994.

[12] M. Driss, Y. Jamoussi, J.-M. Jezequel, and H. H. B. Ghezala.A discrete-events simulation approach for evaluation of

service-based applications. ECOWS’08 : Proceedings of the6th European Conference on Web Services, 2008.

[13] K. Fall and K. Varadhan. The ns manual. Online:http://www.isi.edu/nsnam/ns/doc/ns-doc.pdf.

[14] H. Foster, S. Uchitel, J. Magee, and J. Kramer. Ltsa-ws:A tool for model-based verification of web service com-positions and choreography. in ICSE ’06: Proceedings ofthe 28th International Conference on Software Engineering,pages 771–774, 2006.

[15] M. Huhns and M. P. Singh. Service-oriented computing:Key concepts and principles. IEEE Internet Computing,9(1):75–81, 2005.

[16] E. Mancini, U. Villano, M. Rak, and R. Torella. Asimulation-based framework for autonomic web services.ICPADS ’05: Proceedings of the 11th International Confer-ence on Parallel and Distributed Systems - Workshops (IC-PADS’05), pages 433–437, 2005.

[17] D. A. Menasce. Qos issues in web services. IEEE InternetComputing, 6(6):72–75, 2002.

[18] G. J. Myers. Art of Software Testing. John Wiley & Sons,Inc., New York, NY, USA, 1979.

[19] S. Narayanan and S. A. McIlraith. Simulation, verificationand automated composition of web services. WWW ’02:Proceedings of the 11th international conference on WorldWide Web, pages 77–88, 2002.

[20] M. Nikolaidou and D. Angnostopoulos. An application-oriented approach for distributed system modeling and sim-ulation. ICDCS’01: Proceedings of the The 21st Interna-tional Conference on Distributed Computing Systems, page165, 2001.

[21] L. J. Osterweil. Strategic directions in software quality.ACM Comput. Surv., 28(4):738–750, 1996.

[22] M. P. Papazoglou. Service -oriented computing: Concepts,characteristics and directions. WISE ’03: Proceedings of theFourth International Conference on Web Information Sys-tems Engineering, page 3, 2003.

[23] M. Pistore and P. Traverso. Assumption-based compositionand monitoring of web services. Test and Analysis of WebServices, pages 307–335, 2007.

[24] M. Rouached, O. Perrin, and C. Godart. Towards formalverification of web service composition. Business ProcessManagement, pages 257–273, 2006.

[25] G. Salan, L. Bordeaux, and M. Schaerf. Describing and rea-soningon web services using process algebra. in ICWS ’04:Proceedings of the IEEE International Conference on WebServices, page 43, 2004.

[26] M. Sampath, R. Sengupta, S. Lafortune, K. Sinnamohideen,and D. Teneketzis. Diagnosability of discrete-event systems.IEEE Transactions on Automatic Control, 40(9):1555–1575,1995.

[27] G. Spanoudakis and K. Mahbub. Requirements monitoringfor service-based systems: Towards a framework based onevent calculus. 9th IEEE International Conference on Au-tomated Software Engineering (ASE 2004), pages 379–384,2004.

[28] L. Zeng, B. Benatallah, A. H. Ngu, M. Dumas,J. Kalagnanam, and H. Chang. Qos-aware middlewarefor web services composition. IEEE Trans. Softw. Eng.,30(5):311–327, 2004.