QVM

An Efficient Runtime for Detecting Defectsin Deployed Systems

Matthew ArnoldMartin VechevEran Yahav

IBM Research

Motivation:Dynamic Analysis for Debugging


Testing

High overhead tolerable

Deep properties relating to program correctness


Testing



Production

Low overhead is mandatory

Very limited information

Production

Very limited information

Testing



Production

Low overhead is mandatory

Testing


But Why Modify the VM? VM Disadvantages

Portability Complexity

Why not use bytecode inst., JVMTI, aspects,

java.lang.Instrument ??? VM Advantages

VM only information Free bits in object header Can walk the heap if we desire (GC)

Performance Exploit dynamic optimization technology

Ease of deployment No “install process”. Just set command line flag

New Overhead Philosophy

Traditional dynamic analysis If I use your analysis, how much overhead

will it have? QVM: user specifies an overhead

budget “I am willing to tolerate an X% slowdown” Goal: give user as much useful information as

possible May miss errors

But enables some checking in scenarios where it is currently infeasible

Contributions

Overhead manager (OHM) Adapts analyses to meet user-specified overhead

budget

Dynamic analyses checking correctness properties Typestate property checking

Object-centric sampling Heap probes / assertions Java assertions

QVMI Overhead aware interface for medium-granularity

VM events All implemented and evaluated IBM’s J9 JVM

Execution

Engine

eve

nt

filter

s

event

callbacks

VM Core

Client

s

typestate

client

assertions

client

heap probes

client

QVM

Application

typestate

specsviolations

report

overhead

budget

adjust sampling rates

OHM

observed overhead

QVM Architecture

QVMI

Profiling interface Similar to JVMPI/JVMTI

Method calls, allocations, etc

Key Difference: event filtering

QVMI: The QVM Interface

Execution

Engine

JVMTI

event

filters

event

callbacksVM

Analysis

Client

Profiling interface Similar to JVMPI/JVMTI

Method calls, allocations, etc

Key Difference: event filtering


Execution

Engine

JVMTI

event

filters

event

callbacks

Execution

Engine

QVMI

event

filters

event

callbacksVM

Analysis

Client

Analysis

Client


When compiling a method JIT queries QVM clients

“Does invocation of method foo() require a call back?” If not, no callback is compiled into code

Ensures no overhead for uninteresting events

Overhead Manager (OHM)

OHMExecution

Engine

adjust sampling rates

observed overhead

QVMI

eve

nt

filter

s

event

callbacks

VM Core

specified

overhead

Monitoring: measure overhead incurred by clients Sampling strategy: events callbacks have

adjustable sample rate Controller: adjusts sample rate based on measured

overhead

Overhead Manager Challenges Fine grained timers critical

Read cycle counts via rdtsc instruction Must have notion of “total application

time” Interactive apps

We use Linux getrusage()to get cpu time Issues for multi-threaded apps

Details in paper Analyses must be able to be “turned off”

OK to miss bugs But must not produce meaningless results

Maximizing Sampling Coverage

eventA (…)

eventB (…)

eventC(…)

Execution frequency

Code

Randomly distributed samplingcan produce poor results

Maximizing Sampling Coverage

eventA (…)

eventB (…)

eventC(…)

Execution frequency

1/1

1/1

1/100

SampleRate Code

Origin-specific sampling

QVM Client 1: Typestate Property Checker

undisposed

err

disposedelse

Objectdeath

*

*

b

dispose* | release*

Objectallocation

Typestate Property Checker Simple to implement via QVMI

Events used Object Allocation, method invocation,

object death

Sampling typestate is problematic Ex: File Open Close

High problem of sampling close but not open

Solution: object-centric sampling

Object Centric Sampling

…

T t = new T()

…

tracked tracked

Tracked objects marked using bit in object header

Bit checked before executing callbacks

Client 2: Heap Probes

Heap Probes Allow programmer to query properties of the heap

isShared(Object o1) Do two or more heap objects point to o1

isThreadOwned(Thread t, Object o) Is o reachable from only thread t only

Uses components of a parallel GC to evaluate heap queries Worst case: requires traversal of entire heap Probe sites automatically sampled by overhead

manager Performs GC in process

Use Case Example: Azureus

Over 160 million downloads

Azureus Resource Leaks

QVM ERROR: [Resource_not_disposed] object [0x98837030]of class [org/eclipse/swt/graphics/Image]allocated at site ID 2742 in method[com/aelitis/azureus/.../ListView.handleResize(Z)V]died in state [UNDISPOSED]with last QVM method invoked [org/.../Image.isDisposed()Z].

Typestate checker for undisposed GDI resources

Actual QVM report:

Azureus Resource Leaks

void handleResize(boolean bForce) { // ... if (imgView == null || bForce) { imgView = new Image(listCanvas.getDisplay(),

clientArea); } // …}

imgView

OSResourc

es

OSResourc

es

Possible Fix

void handleResize(boolean bForce) { // ... if (imgView == null || bForce) { if(imgView != null && !imgView.isDisposed()) {

imgView.dispose(); } imgView = new Image(listCanvas.getDisplay(),

clientArea); } // ...}

Possible Fix

void handleResize(boolean bForce) { // ... if (imgView == null || bForce) { if(imgView != null && !imgView.isDisposed()) {

assert(!QVM.isShared (imgView)); imgView.dispose(); } imgView = new Image(listCanvas.getDisplay(),

clientArea); } // ...}

Experimental Evaluation

Overhead Manager: stabilization

Overhead Manager

Leak Detection ResultsApplication

SWT Resources

IOStreams

High Frequency

Fixed

Azureus 11 0 4 5

Etrader 17 0 2 0

Feednread 1 7 0 0

Goim 3 0 1 3

IBMapp1 0 0 0 0

IBM app2 3 2 0 0

Jcommander

9 0 0 0

Juploader 0 1 0 0

nomadpim 2 0 0 0

Rssowl 8 3 0 0

Tvbrowser 0 5 0 0

Tvla 0 4 0 0

Virgoftp 6 0 0 6

Total 60 22 7 14

Sampling coverage (5% budget)

Summary

Recap Adaptive overhead controller Clients: typestate, assertions, heap probes QVMI Found and fixed bugs several real

applications Future Work

Improve efficiency of heap assertions Concurrent or incremental evaluation

Overhead manager Tighter overhead guarantees

Related Work

Much related work in paper Typestate and leak detection Aspect Oriented Programming “Monitoring Oriented Programming” Heap analysis tools Real time garbage collection

The End

Documents

QVM