[email protected]€¦ · Software Maintenance Delta Debugging 4 . Printing this page makes Mozilla crash . Quelle: Zeller, A.; Society, I. C. & Hildebrandt, R.: Simplifying and Isolating

Software Maintenance Delta Debugging

1

Debugging

[email protected]

Delta


2

Outline - Delta Debugging

Motivation

Basic definitions Test result Interference Inconsistency

Algorithms Minimization Differences

Areas of application


3

Motivation

Initial Situation Large Input that causes a failure

Challenge Find a smaller input that still causes the failure

Solution Use the principle of divide and conquer Minimization by Delta Debugging Developed by Andreas Zeller et al.

(www.st.cs.uni-sb.de/dd/)


4

Printing this page makes Mozilla crash

Quelle: Zeller, A.; Society, I. C. & Hildebrandt, R.: Simplifying and Isolating Failure-Inducing Input IEEE Transactions on Software Engineering, 2002, 28, 2002

Excerpt:


5

How do minimize this input?

Web page has over 40,000 characters

Remove single characters 40,000 tests

Better:

First: cut away large parts

Later: increase granularity


6

Narrowing down hypothesis

Source: A. Zeller, www.st.cs.uni-sb.de/dd/

Possible failure cause Set up first hypothesis Test first hypothesis


7



Second hypothesis


8



Third hypothesis


9



Fourth hypothesis


10

<SELECT NAME=„priority“ MULTIPLE SIZE=7> <SELECT NAME=„priority“ MULTIPLE SIZE=7> <SELECT NAME=„priority“ MULTIPLE SIZE=7> <SELECT NAME=„priority“ MULTIPLE SIZE=7>

Minimizing ...

<SELECT NAME=„priority“ MULTIPLE SIZE=7>


11


Minimizing ...

< <SELECT NAME=„priority“ MULTIPLE SIZE=7>


12


Minimizing ...

< <SELECT NAME=„priority“ MULTIPLE SIZE=7> <SELECT NAME=„priority“ MULTIPLE SIZE=7> <SELECT NAME=„priority“ MULTIPLE SIZE=7> <SELECT NAME=„priority“ MULTIPLE SIZE=7> <SELECT NAME=„priority“ MULTIPLE SIZE=7>

. . . <SELECT NAME=„priority“ MULTIPLE SIZE=7>


13

Problem description

Given:

Testing function test(x) mapping input x to ,,.

An input C with test(C) = .

Goal:

Search for a smaller c ⊂ C so that test(c‘) = .


14

Basic definitions Testing function

Takes a program and an input

The test either

succeeds (PASS, ),

produces the failure (FAIL, ) or

produces indeterminate results (UNRES, )


15

DD Process

Test case x

Program

test(x)

Delta Debugging minimize


16

Basic definitions

Interference

Combination of several inputs reveals a bug

No interference: test(x) = iff „1“⊆ x test(x) = otherwise

Interference: test(x) = iff „18“⊆ x test(x) = otherwise


17

Basic definitions

Inconsistency Input a cannot be applied without input b Example C = “12345678” test(x) = ? iff (“2“⊆ x ۸ “7“ ⊄ x) ۷ (“7“⊆ x ۸ “2“ ⊄ x) test(x) = iff “8“⊆ x ⊆ C test(x) = otherwise


18

Basic definitions

=⊆∀ )(: ctestCc ≠⊇∀→ )'(:' ctestcc

=⊆∀ )(: ctestCc ≠⊆∀→ )'(:' ctestcc

• Monotony


19

Algorithm – Version 1

|∆1| ≈ |∆2|


20

Example 1

Input C = “12345678“ Function test test(x) = iff “7“ ⊆ x ⊆ C test(x) = otherwise


21

Example 2

Input C = “12345678“ Function test test(x) = iff “37“ ⊆ x ⊆ C test(x) = , otherwise

Interference


22


Cannot deal with interference!

|∆1| ≈ |∆2|


23


Divide and conquer!


24

Example 2

Input C = “12345678“ Function test test(x) = iff “37“ ⊆ x ⊆ C test(x) = , otherwise


25

Example *

Input C = “12345678“ Function test test(x) = if “18“ ⊆ x ⊆ C test(x) = if “35“ ⊆ x ⊆ C test(x) = otherwise

Several faults


26


Does not work in case of several errors!


27


Instead use the result of


28

Example **

Input C = “12345678“ Function test: test(x) = ? iff (“2“ ⊆ x ۸ “7“ ⊄ x)

۷ (“7“ ⊆ x ۸ “2“ ⊄ x) (case 1)

test(x) = iff “8“ ⊆ x ⊆ C ۸ not case 1 test(x) = otherwise

Inconsistency


29



30

Example 4

Input C = “12345678“ Function test: test(x) = if “34“ ⊆ x ⊆ C test(x) = if “7“ ⊆ x ⊆ C test(x) = otherwise

Minimal solution?


31

Minimality criterias

Global minimum An input c ⊆ C is said to be the global minimum iff

/ Requires testing all 2|C| subsets

Local minimum An input c ⊆ C is said to be the local minimum iff

/ Requires 2|c| -2 tests

|||'|)'(:')( ccctestCcctest <∧=⊆∃∧=

=⊂∃∧= )'(:')( ctestccctest


32

Minimality criterias

The problem of checking local and global minimality is a decision problem which is NP-complete. n-minimality An input c ⊆ C is n-minimal iff

≠⇒≤−⊂∀ )'(|'|||:' ctestncccc


33

Properties of ddmin

ddmin returns a 1-minimal result.

The complexity of ddmin is O(|c|2).

In the best case the complexity of ddmin is Ω(log2(|c|)).


34

bug.c / gcc V2.95.2 (Intel-Linux) #define SIZE 20 double mult(double z[], int n) int i,j; i=0; for(j=0;j<n;j++) i = i + j +1; z[i] = z[i]*(z[0]+1.0);

return z[n]; void copy(double to[], from[], int count) int n = (count + 7 ) / 8; switch (count % 8) do case 0: *to++ = *from++; ...

while (--n > 0); return mult(to,2);

int main (...) ... return copy(y,x,SIZE);

WHICH PART CAUSES GCC TO

CRASH?


35

bug.c / gcc V2.95.2 (Intel-Linux) #define SIZE 20 double mult(double z[], int n) int i,j; i=0; for(j=0;j<n;j++) i = i + j +1; z[i] = z[i]*(z[0]+1.0);

return z[n]; void copy(double to[], from[], int count) int n = (count + 7 ) / 8; switch (count % 8) do case 0: *to++ = *from++; ...

while (--n > 0); return mult(to,2);

int main (...) ... return copy(y,x,SIZE);

Remove half of the program and try gcc again.

If there is a crash, repeat removing

substrings. Otherwise, choose

a different substring.

(BINARY SEARCH)


36

bug.c / gcc V2.95.2 (Intel-Linux) Result after minimization:

double mult(double z[],int n) int i,j; for(;;) i=i+j+1; z[i]=z[i]*(z[0]+1.0); return z[n];

The resulting substring is substantially smaller!


37

Isolation of differences


38

Example 1

Input C = “12345678“ Function test: test(x) = ? iff (“37“⊄ x) (Case 1) test(x) = iff “6“ ⊆ x ⊆ C ۸ not case 1 test(x) = otherwise


39

Areas of application

Test case reduction Minimizing failure inducing input User input Data input

Code reduction Identifying software changes that lead to failures


40

Case Study (Zeller 2002)

Mozilla web browser Example Crash after 95 user actions Automatic reduction to 3 relevant user actions Automatic minimization of 44 bug reports

GCC


41

Conclusions

Successfully used to minimize test cases

Does not (always) find the global/local minimum

Not only for inputs of type STRING

Helps to locate bugs in software


42

Example ***

Input C = “12345678“ Function test: test(x) = ? iff (“26“⊄ x) (Case 1) test(x) = iff “7“ ⊆ x ⊆ C ۸ not case 1 test(x) = otherwise


43

Example ****

Input C = “12345678“ Function test: test(x) = ? iff (“17“⊄ x) test(x) = if “2“ ⊆ x ⊆ C test(x) = if “8“ ⊆ x ⊆ C test(x) = otherwise

Documents

[email protected]€¦ · Software Maintenance Delta Debugging 4 . Printing this page makes Mozilla crash . Quelle: Zeller, A.; Society, I. C. & Hildebrandt, R.: Simplifying and Isolating