RehearsalA framework for automated testing of

web service choreographies

Felipe M. Bessonbesson@ime.usp.br

Masters DefenseAdvisor: Fabio Kon

September 14th, 2012

IME-USP

2

Web services● Interoperable communication

● Machine readable interface

RSS googlecalendar google

places

● In SOA context:● Soap/WSDL web services● A WS can be composed in a more complex one

3

Orchestration

● Centralized● Process● BPEL and SCA

A

● Difficult to manage● Less scalable

B

C

4

Choreography

● Decentralized● Distributed

coordination● WS-CDL and BPMN 2

A

B

C

5

Choreography● Roles: interface (WSDL) and behavior

● Distributed coordinators playing roles (Peltz, 2003)

Store Bank

Shipper

6

Problems of choreographies

● Development● Complex● Focus on modeling● No widely-adopted standards

● Testing● Parallel and many messages● Binding problems● Third-party governance rules

7

Consequences

● A lack of ● development methodologies and processes● tools for automated testing of WS compositions● tools for supporting development (e.g., debugging)

● Consequences● Ad hoc choreographies

– Low quality●Low adoption

8

Our Goals

A testing framework to support Test-Driven Development (TDD) of choreographies

services and roles in isolation

messages exchanged entire

choreography

9

Test-Driven Development (TDD)

● A design technique that drives the development process through testing [1]

RefactorRefactor CodeCode

TestTest

1. Write an AUTOMATED test for next functionality you want to add;

2. Write the functional code until the test passes;

3. Refactor the new and old parts of the code.

[1] (Fowler, 2001; Beck, 2003)

10

Motivation

● Most existing efforts [2] for testing choreographies● Focus on validation and simulation of models● None of them focus on TDD

● Aggregate more importance to development activities

● TDD can facilitate the choreography development and leverage its adoption

[2] (Bucchiarone, 2007; Canfora, 2009; Palacios, 2011)

11

Agenda

● Rehearsal ● TDD methodology proposal● Exploratory study● Conclusions

12

Rehearsal

13

Choreography testing

● Surveys (Bucchiarone, 2007; Canfora, 2009)

Unit

(Peltz, 2003)

Integration Acceptance

14

Unit and Acceptance testing● WSClient: a dynamic generator of web service

clients● Related work

● SoapUI (SmathBear, 2012)● SOCT (Bartolini, 2009) ● BISTWS (Medeiros, 2010)

● Drawbacks:● Tests written in XML ● The service contract must be available

15

WSClient

● Testing web services without having the contract

● Service operations are validated at runtime

16

WSClient

17

WSClientItem explorer

Creating Item objects automatically

18

Integration testing● Message Interceptor● Related work

● Pi4SOA (Pi4, 2012)● CDL-Checker (Wang, 2010)

– Messages are validated by simulation

● BPEL Unit (Mayer, 2006)– Test cases are written in XML– Coupled to BPEL engines

19

Message Interceptor

Intercepting, storing and validating messages exchanged among services and roles

20

Message Interceptor

A BD

C E F

21

Integration testing● How to deal with thirdy-party constraints ?

● Governance rules● Absence of testing environment

● WSMock: service mocking● Related Works

● SoapUI (SmartBear, 2012)– Difficult to manage and customize– Basis for the WSMock implementation

22

WSMock

Simulating regular and faulty scenarios

23

WSMock

24

Abstraction of Choreography

Abstracting the choreography elements into objects● Test cases can be written easily and clearly● Help to understand the choreography execution● Relate the test cases to the choreography models

(diagrams)

BPMN 2 diagrams Rehearsal

Java ObjectsRolesServicesMessages

tests

25

Abstraction of ChoreographyFuture: bookTrip = new Choreography(“book.bpmn2”);

Service airline1 = bookTrip.getServicesByRole(“airline”).get(0);

WSClient ws = new WSClient (airline1.getWSDL());

Item flight = ws.request(“searchFlight”, “São Paulo”, “Paris”, … );

assert(“O-815”, flight.getContent(“id”));

Actual: bookTrip = new Choreography(“descriptor.yml”);

26

TDD Methodology proposal

27

Creation or adaptation of atomic web services

– Choreography developed from scratch● Contract is defined by the tests

– Other cases● Tests guide the development

● Tests are an executable documentation– WSClient!

Phase 01

28

Example of executable specification

Phase 01

29

Phase 2: services to compose the roles

Phase 3: roles to compose the choreography

– Validate the messages exchanged inside/outside the executable processes

● Mocking real dependencies● Using the Message Interceptor for

inspecting and validating the messages

– Detect binding problems early

Phases 2 and 3

30

Acceptance and Scalability Testing– Validate all choreography features from the

user point of view● Trying to use an environment closer to the

production

– Assess the choreography scalability● large workload and different choreography

configurations

Phase 4

31

Exploratory Study

32

● Subjects: 21 CS students of IME-USP● 2 phases

● Varying the experience in web service development and TDD

● Systematic protocol

Design

( 4 tasks )

33

RQ1: Does the Rehearsal features aid in the application of the proposed methodology steps ? RQ2: Does the proposed methodology provide adequate guidelines for developing a choreography ?

Phase 1

Assessing the adequacy of using Rehearsal following the TDD methodology proposal

34

Experience # 8 studentsp

arti

cip

ants

par

tici

pan

ts

par

tici

pan

ts

par

tici

pan

ts

par

tici

pan

ts

Software development (years) TDD (years) Java (years)

WS development (years) WS compositions (years)

1-3 3-5 more than 5 0-2 more than 3 1-3 3-5 more than 53-5

0-10more than 21-20-1

35

RQ1: How easy is to use the Rehearsal features to apply the methodology steps ?

RQ2: How easy is to follow the methodology steps to develop a choreography ?

Phase 2

Assessing how easy is to use and learn Rehearsal and the TDD methodology

36

Experience # 11 students

par

tici

pan

tsp

arti

cip

ants

par

tici

pan

tsp

arti

cip

ants

Software development (years) TDD (years)

WS development (years) Java (years)

1-3 3-5 more than 50 0-1 1-2

1-2 more than 20-10 1-3 more than 5

37

Example of Question

The questions of each statement was provided following the scale:

5. Strongly Agree

4. Partially Agree

3. Indifferent

2. Partially Disagree

1. Strongly Disagree

38

Qualitative Results

Percentile(25,75) of QtiPlot

39

Rehearsal – WSClient● QR1: It was easy to learn how to use the WSClient feature● QM3 (p1): The use of the WSClient feature is useful in Task 01● QM3 (p2): It was easy to use the WSClient feature in Task 01● QM13 (p1): The use of the WSClient feature is useful in Task 04● QM13 (p2): It was easy to use the WSClient feature is useful in Task 04

questions - phase 1 questions - phase 2

40

Rehearsal – WSMock● QR2: It was easy to learn how to use the WSMock feature● QM6 (p1): The use of the WSMock feature is useful in Task 02● QM6 (p2): It was easy to use the WSMock feature in Task 02

questions - phase 1 questions - phase 2

41

Rehearsal – Abstract of choreography● QR4: It was easy to learn how to use the Abstraction Choreography

● QR5: The Abstraction Choreography feature helped me to use other Rehearsal features

● QR6: The Abstraction Choreography feature helped me to write the test cases

questions - phase 1 questions - phase 2

42

Methodology – Acceptance● QM14: I think the use of the methodology and Rehearsal would be

useful for projects I have participated● QM15: I would use the methodology and Rehearsal in future

projects I may participate.

questions - phase 1

43

Contributions

● An Open Source Testing Framework● Examples of web service choreographies● Contributions to the SoapUI community● Exploratory study package

● Protocol● Training material● Results ● Produced code

44

Contributions

Felipe M. Besson, Pedro M. B. Leal, Fabio Kon, Alfredo Goldman and Dejan Milojicic. Towards automated testing of web service choreographies. 6th International Workshop on Automation of Software Test (AST'11) on ICSE. Waikiki, Honululu, HI, USA, 2011.

Felipe M. Besson, Pedro M. B. Leal, Fabio Kon, Alfredo Goldman and Dejan Milojicic. Supporting Test-Driven Development of Web Service Choreographies. 5th Open Cirrus Summit, Moscow, Russia, 2011.

Felipe M. Besson, Paulo Moura, Fabio Kon, Dejan Milojicic. Rehearsal: A framework for automated testing of web service choreographies. 3th Brazilian Conference in Software Theory and Pratice, Tool Session, Natal, Brasil, 2012. (to be presented)

45

Future Work

Improve Rehearsal functionalities● Abstraction of choreography● Dynamic reconfiguration● Generate test cases automatically● Testing coverage

Improve TDD methodology● Integrate to choreography modeling

Software Engineering Experiments● Compare with ad hoc development

methodologies

46

Conclusions

Our goals have been reached =]● Good results in the exploratory study

– For experienced and non experienced developers

● Relevant contribution to the topic (academy and industry)

Rehearsal and TDD methodology can aggregate more importance to the developer role in the life cycle of SOC applications

47

Thanks a lot !!!

● My parents ● My girlfriend Jaq S2 ● My advisor Fabio ● Friends of IME-USP● Friends of Elo7

48

Thanks to

● HP Brasil under the Baile Project● European Community's Seventh Framework Program

FP7/2007-2013 under grant agreement number 257178 (project CHOReOS - Large Scale Choreographies for the Future Internet)

Acknowledgements

Specially to Dejan Milojicic

49

Chris Peltz. Web Services Orchestration and Choreography. Computer, 36:4652, October 2003.

Martin Fowler. Test-Driven Development. Available on: http://www.martinfowler.com/bliki/TestDrivenDevelopment.html, 2011.

Kent Beck. Test-driven development: by example. Addison-Wesley, Boston, 2003.

Antonio Bucchiarone, Hernán Melgratti, and Francesco Severoni. Testing Service Composition. In 8th Argentine Symposium on Software Engineering (ASSE'07), Mar del Plata, Argentina, 2007.

Gerardo Canfora and Massimiliano Di Penta. Service-Oriented Architectures Testing: A Survey. In Software Engineering, volume 5413 of Lecture Notes in Computer Science, 2009.

Marcos Palacios, José García-Fanjul, and Javier Tuya. Testing in Service Oriented Architectures with dynamic binding: A mapping study. Information and Software Technology, pages 171189, March 2011.

References

50

SmartBear. SoapUI, Web Services Functional Testing Tool. Available on: <http://http://www.soapui.org/>, 2010.

Cesare Bartolini, Antonia Bertolino, Sebastian Elbaum e Eda Marchetti. Whitening SOA testing. In Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT Symposium on The foundations of software engineering, ESEC/FSE '09, New York, NY, USA, 2009.

Marcelo Medeiros Eler, Marcio Eduardo Delamaro, Jose Carlos Maldonado e Paulo Cesar Masiero. Built-In Structural Testing of Web Services. Em Proceedings of the Brazilian Symposium on Software Engineering, SBES '10. IEEE Computer Society, 2010

Pi4 Technologies Foundation. Pi calculus for SOA. Available on: http://sourceforge.net/projects/pi4soa/, 2010.

San-Yih Hwang, Wen-Fan Hsieh e Chien-Hsiang Lee. Verifying web services in a choreography environment. Em SOCA. IEEE, 2011.

Philip Mayer e Daniel Lübke. Towards a BPEL unit testing framework. Em Proceedings of the 2006 workshop on Testing, analysis, and verification of web services and applications, TAV-WEB '06, New York, NY, USA, 2006

References