5/7/03 ICSE 2003 1

Fragment Class Analysis for Testing of Polymorphism in Java Software

Atanas (Nasko) RountevOhio State University

Ana MilanovaBarbara Ryder

Rutgers University

5/7/03 ICSE 2003 2

Testing of Polymorphism

Testing of object-oriented software Inheritance, polymorphism, object state, etc.

Polymorphism A variable may refer to objects of different

classes

Existing work on testing of polymorphism Basic idea: exercise all possible polymorphic

bindings Our focus: bindings at polymorphic calls

5/7/03 ICSE 2003 3

Two Coverage Criteria for Calls

RC criterion: exercise each call site with all possible classes of the receiver object At x.m(), x may refer to A, B, or C objects

TM criterion: exercise each call site with all possible target methods x.m() may invoke A.m or C.m

Goal: build a tool for Java that measures RC/TM coverage achieved by test suites The approach is also applicable to other

criteria for testing of polymorphism

5/7/03 ICSE 2003 4

Coverage Tool for Testing of Polymorphism

Step 1: analyze the tested code What are all possible receiver classes and

target methods at polymorphic call sites? Need class analysis

Step 2: insert instrumentation

Step 3: run tests and report coverage What were the receiver classes and target

methods actually observed while running the tests?

comparecompare

5/7/03 ICSE 2003 5

Outline

Class analysis Class analysis of partial programs

Contribution 1: General approach that enables existing analyses to handle partial programs

Analysis imprecision Experimental results

Contribution 2: Determining the imprecision of four popular class analyses

Conclusions and future work

5/7/03 ICSE 2003 6

Class Analysis

Essential static analysis for object-oriented languages; large body of existing work

Given a reference variable x, what kinds of objects may x refer to? e.g. “x may refer to instances of A, B, or C”

Uses of class analysis Compiler optimizations: e.g. devirtualization Software understanding and restructuring Testing: e.g. to compute the RC/TM criteria

5/7/03 ICSE 2003 7

Problem 1: Partial Programs

Many existing class analyses Typically defined as whole-program analyses Cannot be used directly when testing is done

on partial programs

Analysis input Component under test (CUT): set of classes CUT interface that is being tested

Methods and fields against which tests have been written

Server classes of CUT classes

5/7/03 ICSE 2003 8

Fragment Class Analysis

Goal: adapt whole-program class analyses Solution: fragment class analysis

Step 1: Create additional code that “simulates” the possible effects of unknown external code Located inside an artificial main

Step 2: Run a whole-program class analysis starting from main Use the resulting solution to determine RC and

TM at polymorphic call sites in the CUT

5/7/03 ICSE 2003 9

Analysis Solution

If it is possible to write a test that executes x.m() with a receiver of class Y, the analysis solution must report Y Need to “simulate” all possible (infinitely many)

tests for the component

This property provably holds for a large category of whole-program class analyses Flow insensitive Context insensitive and context sensitive

5/7/03 ICSE 2003 10

Placeholder Variables

main contains placeholders for all reference variables from unknown external code

class A { public A() {...} public Y m(Z z) {...}}

class B extends A { public B m2(A a) {…}}

Placeholder variablesmain() { A ph_a; B ph_b; Y ph_y; Z ph_z; …}

CUT

5/7/03 ICSE 2003 11

Placeholder Statements

class A { public A() {...} public Y m(Z z) {...}}

class B extends A { public B m2(A a) {…}}

Placeholder statementsmain() { … ph_a = new A(); ph_y = ph_a.m(ph_z); ph_b = ph_b.m2(ph_a); … ph_a = ph_b; ph_b = (B) ph_a; …}

5/7/03 ICSE 2003 12

Placeholder Statements

class A { public A() {...} public Y m(Z z) {...}}

class B extends A { public B m2(A a) {…}}

Placeholder statementsmain() { … ph_a = new A(); ph_y = ph_a.m(ph_z); ph_b = ph_b.m2(ph_a); … ph_a = ph_b; ph_b = (B) ph_a; …}

5/7/03 ICSE 2003 13

Placeholder Statements

class A { public A() {...} public Y m(Z z) {...}}

class B extends A { public B m2(A a) {…}}

Placeholder statementsmain() { … ph_a = new A(); ph_y = ph_a.m(ph_z); ph_b = ph_b.m2(ph_a); … ph_a = ph_b; ph_b = (B) ph_a; …}

5/7/03 ICSE 2003 14

Problem 2: Analysis Imprecision

Among all classes reported by the analysis, which ones are impossible at run time? Coverage requirements that cannot be satisfied

Goal: determine the imprecision of four fragment class analyses, derived from: Class Hierarchy Analysis (CHA) Rapid Type Analysis (RTA) 0-CFA Andersen’s points-to analysis (AND)

No such data is available in previous work

5/7/03 ICSE 2003 15

Experiments

Considered 8 CUTs From publicly available Java libraries CUT size: between 8 and 24 classes

Wrote tests that achieve as much RC/TM coverage as possible Two people wrote tests independently Compared differences and merged

Metric of imprecision: coverage reported by the tool for these tests Report of 100% means no analysis imprecision

5/7/03 ICSE 2003 16

RC Coverage

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8CUT

Cov

erag

e [%

]CHA RTA 0-CFA AND

5/7/03 ICSE 2003 17

Results

Bad news: CHA and RTA have significant imprecision and should be avoided

Good news: 0-CFA and AND have much lower imprecision Close to 100% for 5 of the 8 components Good candidates for future investigations

Open questions Will we see the same trend for other CUTs? What is the improvement for analyses that are

theoretically more precise than 0-CFA and AND?

5/7/03 ICSE 2003 18

Future Work

More datapoints More class analyses (e.g. context-sensitive) When do analyses fail?

Common sources of imprecision

Generalize to flow-sensitive analyses Apply to existing test suites

Measure achieved coverage Identify weaknesses and add more test cases