Evaluating and Tuning a Static Analysis to Find Null Pointer Bugs

David Hovemeyer, Jaime Spacco, and William Pugh

Presented by Nathaniel AyewahCMSC838P

11/16/2006

Why Simple

• Programmers make simple mistakes// org.eclipse.jdt.internal.ui.compare.JavaStructureDiffViewer

Control c = getControl();

if (c == null && c.isDisposed())

return;

• Low False Positive Rate • Cannot find all bugs

Findbugs

Findbugs

INPUTINPUT PROCESSINGPROCESSING OUTPUTOUTPUT

• Set of “.class” files containing byte-code

• Configurations

• Bug Pattern Code

• Source Line Number

• Descriptive Message

Detectors

Findbugs Detectors

PROCESSINGPROCESSING• Independent of each other

• May share some resources

• GOAL: Low false positives

• Each detector is driven by a set of heuristics

Know Your Know Your Bug PatternsBug Patterns

Output

HIGHHIGHSEVERE RISK OF

PROGRAM FAILURE

MEDIUMMEDIUMELEVATED RISK OF PROGRAM FAILURE

LOWLOWLOW RISK OF

PROGRAM FAILURE

Source: US Department of Program Security

Findbugs Detectors

PROCESSING

Null Pointer Analysis

Null Pointer Analysis

PROCESSINPROCESSINGG• Forward intra-procedural

• Build Control Flow graph for each method

Data-flow FrameData-flow Frame

Method parmeter, local variable, or stack operand

NullNonNull

Slot

Simple Analysis

Detector foo = null;

foo.execute();

Dereferencing Dereferencing NullNull

Detector foo = new Detector(…);

foo.execute();

Dereferencing Dereferencing NonNullNonNull

HIGHHIGHSEVERE RISK OF

PROGRAM FAILURE

If only it were that simple…

• Is a method’s parameter null?

void foo(Object obj) {

int x = obj.hashcode();

…

}

• Infeasible Paths

Infeasible Paths

• Guard indirectly connected to null check

boolean b;

if (p != null)

b = true;

else

b = false;

if (b)

p.f()

Infeasible Paths

• Assertions

p = null;

...

// throws exception if p null:

checkAssertion(p != null);

p.f(); // safe

Infeasible Paths

• Checked Exceptions that are never thrown

Foo dup = null;

try {

dup = super.clone();

} catch (CloneNotSupportedException e) {

// Can’t happen

}

dup.contents = ...

Solution

• NullNull and NonNullNonNull are not enough

No Kaboom No Kaboom NonNullNonNull

Checked Checked NonNullNonNull

NonNullNonNull

NCPNCPNull-ENull-E

NSP-ENSP-ENullNull

NSPNSP

if (b) {A

} else {B

}

C ?

Solution

• Dereferencing a variable that has value Null, NSP, …

Null-ENull-E

NSP-ENSP-ENullNull

NSPNSP

…

HIGHHIGHSEVERE RISK OF

PROGRAM FAILURE

MEDIUMMEDIUMELEVATED RISK OF PROGRAM FAILURE

LOWLOWLOW RISK OF

PROGRAM FAILURE

MEDIUMMEDIUMELEVATED RISK OF PROGRAM FAILURE

Solution

• Choosing a value for a variable after each statement:

Statement Value of pp = null Nullp = this NonNullp = new ... NonNullp = "string" NonNullp = Foo.class NonNullp = q.x NCPp = a[i] NCPp = f() NCP

Solution: Infeasible Paths

p = null;

Null

Null

p = new …

NonNull

NSP

Null

NCP NCP

Solution: Infeasible Paths

Null or NSP

checkAssertion(p != null)

NCP

p.f()

Solution: Infeasible Paths

try {

}

catch(Exception e) {

}

Null NSP

Null-E NSP-E

Comparing a Value to null

foo.execute();

if (foo != null) {

...

}

Comparing Comparing No-KaboomNo-Kaboom to to

nullnull

HIGHHIGHSEVERE RISK OF

PROGRAM FAILURE

Comparing a value to null

Detector foo = null;

if (foo != null) {

foo.execute();

}

Comparing Comparing NullNull to nullto null

if (foo != null) {

...

if (foo == null) {

foo = new ...

}

}

Comparing Comparing Checked Checked

NonNullNonNull to null to null

MEDIUMMEDIUMELEVATED RISK OF PROGRAM FAILURE

R.I.P

MEDIUMMEDIUMELEVATED RISK OF PROGRAM FAILURE

R.I.P

Other Solutions

• Check for methods that unconditionally dereference parameters

• Annotations– @NotNull: parameter/return

value must not be null– @CheckForNull: check the

parameter/return value before dereferencing it

Experiments: Student Code

• With Annotations

Project NPE Warning FN %

Search TreeWeb Spider

71162

38127

4621

Project Warning NPE FP %

Search TreeWeb Spider

40129

36101

1021

Experiments: Student Code

• Without Annotations

Project NPE Warning FN %

Search TreeWeb Spider

71162

147

9870

Project Warning NPE FP %

Search TreeWeb Spider

277

275

02

Experiments: Production Code

• Cannot calc. false negatives!

Warning type Serious False FP %

Null dereferenceNo-Kaboom RCNOther RCN

733315

161517

183153

Eclipse 3.0.1

Conclusion

• More inter-procedural techniques could find more bugs

• But often finding simple bugs with low FP rate is effective