Precisely Selecting Regression Test for Aspect-Oriented ProgramsGuoqing Xu
The Ohio State [email protected]
Outline Big picture and background Problem statement and motivation Our analysis Implementation status Related work
Big Picture AOSD improves separation of concerns Challenges for analysis of AO programs
Control flow in bytecode is not equivalent to the program logic in source code
Compiler-specific weaving Dynamic/static parts of pointcut designators No analysis framework implementation available
Challenges for testing of AO programs Coverage criteria; test generation for exercising of
aspects; regression test selection
Big Picture Long-term goals
Framework that supports static analyses similar to popular analyses for OO program
Develop AO-specific techniques for analysis and testing
Topic of this talk Regression test selection for AO programs Work in progress
Regression Test Selection Testing after software modifications Re-running the entire regression test suite is
expensive Select a subset of tests to run Safe test selection chooses every test case
that may reveal a fault
Related Previous Work Rothermel and
Harrold, TOSEM 97 Graph traversal
algorithms Harrold et al.,
OOPSLA 01 Java interclass
graph (JIG) Copied from [Harrold et al., OOPSLA 01]
Execute P/Record
Coverage
IdentifyDangerous
Entities
Select Tests
Program P
Program P’
Program P
Program P’
P’s Edge Coverage Matrix
Dangerous Entities
containing edges in
P
Regression Testing for AOSD When AO features are added/modified, the
program needs to be regression tested Case 1: P is an OO program and P’ is an AO
version of P Case 2: Both P and P’ are AO programs How should regression test selection be How should regression test selection be
performed for AO software?performed for AO software?
Outline Big picture and background Problem statement and motivation Our analysis Implementation status Related work
Existing Work Applied to AO Programs How to compare two JIGs?
The JIG of the woven code includes redundant nodes and edges and does not correspond to the logical control flow as presented in the source
Need new representations How to recover CFG edges from the
execution trace when computing edge-coverage matrix? (when P is AO program) The execution trace is compiler-specific Need instrumentation before/during weaving
Existing Work Applied to AO Programs1. class C {
2. public void m(int i) {…}3. }4. 5. void bar(C p, int k) {6. p.m(k);7. }Java version P
CFG edge Call edge
Java Interclass Graph (JIG)
bar()
p.m()
return
C.m()
…
exit exit
Example1. class C {2. public void m(int i){…};3. }4. 5. void bar(C p, int k) {6. p.m(k);7. }8. 9. aspect Sample{10. void around(C c, int i):11. call(C.m(int)) && 12. target(c) &&13. args(i) {14. proceed(p , x);15. }
bar()
p.m()
Return
C.m()
…
exit
JIG for P AspectJ Version P’
C.m ()
…
exit
bar()
..around$0
return
…
c.m()
return
JIG for P’ (from woven code)
..Sample
.around$0
Some Results When we applied the [Harrold et.al. 01]
algorithm to several subjects:
Subjects V1 (java version)Num of tests
V2 (AspectJ version)Num of selected tests
bean 10 10
introduction 40 40
observer 10 10
telecom 30 30
tjp 10 10
tracing 40 40
testReadFile 20 19
Possible Approach Create “clean” CFGs in which the wrapper
code inserted during weaving is removed Graph traversal and comparison corresponds to
the “logical” structure of the code, not the compiler-specific woven code
New representation: AJIG AspectJ Inter-module Graph – more later
For regression test selection, need to consider additional issues
A More Complex Problem1. class C {2. public void m(int i){…};3. }4. 5. void bar(C p, int k) {6. C.m(k);7. }8. 9. aspect Sample{10. void around(C c, int
i):11. call(C.m(int)) && 12. target(c) &&13. args(i) {14. System.out.println(i);15. proceed(p , x);15. }
C.m ()
bar()
return
m()
AJIG for P’
…
exit exit
System.out….
Do we need to select all the tests that go through the edge marked in red?
Why it needs to be addressed This is an issue not only for AO software, but
also for procedural and OO software Advices are often free of side effects
Study in [Rinard et al. FSE 04] reported 6 “observer” advices out of ten inspected advices
Recommended for “safe” AO programming Adding side-effect-free advices should not
result in overly conservative regression test selection
Approach: use side-effect analysis
Outline Big picture and background Problem statement and motivation Our analysis Implementation status Related work
Our Work Consider both situations:
Case 1: P is an OO version, P’ is an AO version Case 2: both P and P’ are AO versions
Analysis to select regression tests Build a new control flow representation: AJIG Apply existing graph-traversal algorithm on AJIG Side-effect analysis when comparing AJIGs
AJIG AspectJ Inter-module Graph (recent work)
For the Java parts, same as JIG. Shadow node A shadow node is associated with
a set of JIGs of advices the precedence of these advices an integrated shadow advice JIG
AJIG supports all static AspectJ join point types Conservatively approximate the dynamic part of pointcut
designators.
Side Effect Side effect node
Mutate the objects that have existed before the method/advice is invoked
For an advice, All the nodes in the CFG path that does not contain a
proceed node The proceed node has side effect if its actual
parameters and parameters of the advice don’t point to same objects.
Mutates the object returned by the proceed node The return node has side effect if the return value of
the advice and that of the proceed node don’t point to the same object
Side Effect (Cont’d) Side effect related node in AJIG
Has side effects Has some dependency on the nodes that have side
effects Safe edge in AJIG
An edge is safe edge, if the sink node of this edge is not side effect related.
New Test Selection criterion New Test Selection criterion:
Dangerous set S computation: for each edge e in P, and its counter part e’ in P’ e is not equivalent to e’ and Neither e nor e’ is a safe edge.
Select a test that execute one or more edges in S
Selection
Computing dangerous set S by comparing AJIGs.
We plan to use some form of side-effect analysis Large body of existing work
Selecting tests
Implementation progress
The implementation of algorithms described in [Harrold et.al. 01] √
Building AspectJ Inter-module Graph √ Make an extension for the abc compiler that
generates the Jimple based CFGs for aspects between the weaving of inter-type declarations and advices.
Instrument advices at different phases. Side effect analysis ongoing work Evaluation
Evaluation Plan Benchmarks: ten to twenty moderate size (50-100
classes) AO projects taken from tmbenches, used by the abc compiler.
Experiments collect data.
The number of tests selected over versions. The comparison between enabling side-effect
analysis and disabling it.
Related Work Static/Dynamic Analysis for AO programs.
abc compiler [AOSD 05] [PLDI 05] [TR 04]. Static analysis of aspects [Sereni and Moor,
AOSD 02]. Zhao’s work on the analysis and testing of AO
programs [COMPSAC 03] [WPC 02][AOSD 06]. Classification system for AO programs [Rinard et.
al. FSE 04].
Related Work (Cont’d) Regression Test Selection
[LW ICSM 91], [CRV ISCE 94], [RH TOSEM 97], [Ball ISSTA 98], [Harrold+ OOPSLA 01], [OSH FSE 04]…
Change Impact Analysis [KGH+ ICSM 94], [RT PASTE 01], [OAH FSE 03],
[OAL+ ICSE 04], [RST+ OOPSLA 04], …
Thank you!!
Questions??