Testing AOPsTesting AOPs

CS 5382 Project Presentation

Carlos Acosta

Instructor Dr. Cheon

Testing AOPs - Carlos Acosta

ProblemProblem

AOP provides many advantages (e.g modularity, separation of crosscutting concerns, locality of change)

Testing: an essential part in Software Development Process

Characteristics of AOP makes testing AOPs a challenging issue that is still being researched

Relatively few research on testing AOPs

Testing AOPs - Carlos Acosta

Project GoalsProject Goals

Explore the current research problems or issues in testing AOPs– Compare/analyze testing techniques

available in other programming paradigms (e.g. testing OOPs)

– Compare/analyze existing criteria and techniques for testing AOPs

Testing AOPs - Carlos Acosta

BackgroundBackground

Testing– Software Testing

• Any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results.

– Purpose• Improve quality (conformance to the specified design

requirement )• For Verification (deals with building the model right) and

Validation (deals with building the right model).• For reliability estimation

– Taxonomy

Testing AOPs - Carlos Acosta

Background-2Background-2

Aspect Oriented Programming– Concern– Crosscutting Concern– Join Point– Pointcut– Advise– Aspect– Weaving

Testing AOPs - Carlos Acosta

Surveyed WorkSurveyed Work Surveyed work: Towards the Systematic Testing of Aspect-

Oriented Programs– Problem addressed:

• This paper introduces a candidate fault model, along with associated testing criteria, for AOPs based on interactions that are unique to AOPs. The paper also identifies key issues relevant to the systematic testing of AOPs.

Analysis of surveyed work 1– Contribution:

• Fault model, testing criteria, and identify key issues– Approach:

• Analyze interactions that are unique to AOPs– Issues:

• Strength in pointcuts, advice precedence, failure to establish expected postconditions, incorrect focus of control flow, preserve state invariants, incorrect changes in control dependencies

Testing AOPs - Carlos Acosta

Analysis of Surveyed Work 1 (cont)Analysis of Surveyed Work 1 (cont)

– Conclusion: • The combination of sound AOP construction methods and

systematic AOP testing is an important step in broad acceptance of AOP as a software development approach.

– Future work: • Validate and further define this fault model. Answers to the

following open questions– Are there ways to test aspects on their own?– Can we reverse engineer the weave process?– Can we measure test coverage after weaving?– How do we test aspects that interact with a core concern?– How do we test aspects that mutually interfere?– How do we test aspects whose effects must span more than

one concern?

Testing AOPs - Carlos Acosta

Surveyed Work-2Surveyed Work-2 Surveyed work: UnitTesting Aspectual Behavior

– Problem addressed:• Focus on testing aspectual behavior, i.e. behavior implemented in pieces of

advice. • JamlUnit shows that performing unit testing on aspectual behavior is

possible Analysis of surveyed work 2

– Contribution:• It shows the possibility and requirements for devising clean unit testing

techniques for aspect-oriented programs using mock object mechanisms – Approach:

• JamlUnit uses mock objects that emulate execution context (join point) information.

– Issues:• Criteria to select meaningful mock points have not been developed• Creating mock execution context. The difficulty is inherent to the nature of

aspects, especially for those that interact with core concerns.

Testing AOPs - Carlos Acosta

Analysis of Surveyed Work 2 (cont)Analysis of Surveyed Work 2 (cont)

– Conclusion:• JamlUnit, an extension of JUnit, is proposed as a

framework for performing unit testing of aspects written in JAML

• Performing unit testing on aspectual behavior is possible and relatively straightforward, as long as the AOP language observes a couple of simple requirements.

– Future work:• Providing support tools for writing mock execution

context• Unit-test the XML binders, i.e. the pointcut

definitions• Devising techniques for testing other elements of

JAML, including aspectual bindings and introductions

Testing AOPs - Carlos Acosta

Surveyed Work-3Surveyed Work-3 Surveyed work: A State-Based Approach to Testing Aspect-

Oriented Programs– Problem addressed:

• Presents an approach to testing AOPs: State-Based Analysis of surveyed work 2

– Contribution:• It shows the possibility to generate test suites for adequately testing object

behavior and interaction between classes and aspects in terms of message sequences

– Approach:• Based on FREE (Flattened Regular Expression)-based test design pattern

for OOP. – FREE is similar to the state model in UML

• Extend FREE to ASM (Aspectual State Models) to specify classes and aspects

• Transform ASM transition tree, where each path from the root to a terminal leaf node becomes a test case

– Issues:• State-Based approaches often suffer from the state explosion problem

Testing AOPs - Carlos Acosta

Analysis of Surveyed Work 3 (cont)Analysis of Surveyed Work 3 (cont)

– Conclusion:• A state-based testing approach is possible for AOPs• The use of this testing approach reveals faults such

as: incorrect strength of pointcut patterns and failure to preserve state invariants

– Future work:• How to identify the paths that are of most interest or

importance? (state explosion problem)• How the testing approach fits in other UML-based

modeling methods for AOSD• Extend the approach for revealing likely faults in

composition of crosscutting concerns

Testing AOPs - Carlos Acosta

Related WorkRelated Work

A Survey of Testing Techniques for Object-Oriented Systems– Survey of research in testing object-oriented systems. – Discusses the challenges to testing an OO system, the

different levels of verification, and various testing techniques Testing of Object-Oriented Programming

Systems (OOPS): A Fault-Based Approach– Examines the testing of object-oriented systems, compares,

and contrasts it with the testing of conventional programming language systems, with emphasis on fault-based testing

Testing AOPs - Carlos Acosta

ConclusionsConclusions Software Testing is challenging and is important throughout the software

development process. Previous surveys on testing OOPs reveals a survey on testing AOPs is

useful and is a great means of presenting current problems and approaches as well as future work on testing techniques for AOPs.

Relatively few research on testing AOPs. Current research on:

– Developing and Improving fault models for testing AOPs – Developing testing criteria for testing AOPs– Developing frameworks, such as JamlUnit, for performing unit testing of

aspects written in JAML– Data-Flow-based approach for Unit testing AOPs – State-based approach for testing AOPs– Model-based approach to test generation for AOPs– Testing AOPs based on specifications

There are still many open questions and issues that have not been addressed on testing AOPs

Testing AOPs - Carlos Acosta

ReferencesReferences Balci, Osman. Principles and techniques of simulation validation, verification and testing. in: Alexopoulos, C.;

Kang, K.; Lilegdon, W. R., and Goldsman, D. Proceedings of the 1995 Winter Simulation Conference; 1995:147-154. Cristina C. Lopes and Trung C. Ngo. Unit-Testing Aspectual Behavior. In Proceedings of Workshop on Testing

Aspect-Oriented Programs (WTAOP), held in conjunction with the 4th International Conference on Aspect-Oriented Software Development (AOSD), 2005. To appear.

D. Xu, W. Xu, and K. Nygard. A State-Based Approach to Testing Aspect-Oriented Programs. Technical Report NDSU-CS-TR04-XU03, North Dakota State University Computer Science Department, September 2004.

Dianxiang Xu. Test Generation from Aspect-Oriented State Models. Technical Report, NDSU-CS-TR-05-XU02, Sept 2005.

J. Zhao. Data-Flow-Based Unit Testing of Aspect-Oriented Programs. In Proc. 27th Annual IEEE International Computer Software and Applications Conference (COMPSAC'2003), pages 188.197, Dallas, Texas, December 2003.

Jane Huffman Hayes, "Testing of Object-Oriented Programming Systems (OOPS): a Fault-Based Approach," Proceedings, Object-Oriented Methodologies and Systems, E. Bertino, S. Urban (Eds.), LNCS 858. Springer-Verlag, Berlin 1994.

Jiantao Pan. Software Testing. 18-849b Dependable Embedded Systems, Carnegie Mellon University, Spring 1999. Morris S. Johnson, Jr. A Survey of Testing Techniques for Object-Oriented Systems. Proceedings of the 1996

conference of the Centre for Advanced Studies on Collaborative research Page: 17 , Toronto, Ontario, Canada, 1996 R. Alexander, J.M. Bieman, and A. A. Andrews. Towards the Systematic Testing of Aspect-Oriented Programs.

Technical Report CS-4-105, Colorado State University, 2004. T. Xie, J. Zhao, D. Marinov, and D. Notkin, "Automated Test Generation for AspectJ Programs", AOSD 2005

Workshop on Testing Aspect-Oriented Programs, Chicago, USA, March 2005. Xu, W. and Xu, D. A model-based approach to test generation for aspect-oriented programs. AOSD 2005 Workshop

on Testing Aspect-Oriented Programs, Chicago, March 2005.