22
November 2005 New Features in Paradyn and Dyninst Matthew LeGendre [email protected] Ray Chen [email protected]

November 2005 New Features in Paradyn and Dyninst Matthew LeGendre [email protected] Ray Chen [email protected]

Embed Size (px)

Citation preview

November 2005

New Features in Paradyn and Dyninst

Matthew [email protected]

Ray [email protected]

– 2 –

The Road from 4.2 to 5.0 Paradyn/Dyninst 4.2 was released in March

2005. Paradyn/Dyninst 5.0 coming very soon.

In that time, we’ve changed ~40,000* lines out of Dyninst’s 146,000 line code base.• But the code base only grew by ~4,000 lines

* loose estimate

– 3 – Dyninst 5.0

Major Features MRNet integration into Paradyn More powerful instrumentation system New threaded event handler

Support for multithreaded mutatees Performance improvements New test suite Linux/AMD64 support Improved Windows support

– 4 – Dyninst 5.0

Improvements in Scalability Paradyn/Dyninst 5.0 will be able to handle

a wider variety of applications at a more fine-grained level.

We recently used Dyninst to track every branch and return instruction in the Microsoft Word executable.

Users have used pre-releases to instrument every basic block in a ~200MB binary.

– 5 – Dyninst 5.0

MRNet Integration Paradyn 5.0 now uses MRNet as a

communication mechanism between the front end and its backends.

Lays the groundwork for us to start using Paradyn in large-scale computing environments.

An experimental implementation easily ran Paradyn on a 1024 node job.

– 6 – Dyninst 5.0

New Parsing and Instrumentation Systems

New parsing system helps us to distinguish code from data, allowing for more reliable and safe instrumentation.

New instrumentation system allows us to efficiently instrument any instruction in a binary.

– 7 – Dyninst 5.0

Support for Multithreaded Apps Allows Dyninst to work with multithreaded

mutatees•Linux/x86, Linux/IA64, Linux/AMD64,

AIX/Power, Solaris/SPARC, and Win32/x86

Mutators will need to be upgraded to use the new threading interface before they’ll work on multithreaded mutatees.

– 8 – Dyninst 5.0

BPatch_process

Current DyninstAPI Model

BPatch_thread

BPatch_image

BPatch_module

BPatch_function

– 9 – Dyninst 5.0

Backwards Compatibility Old Dyninst mutators will continue to work

on single threaded applications.

Process operations that used to exist in BPatch_thread are being cloned in BPatch_process.

For Example:• BPatch_thread::malloc will call BPatch_process::malloc in the owner process.

– 10 – Dyninst 5.0

Performance Improvements Lazy parsing of binaries and libraries

speeds up Dyninst startup.

Cleaned up memory leaks and reduced memory usage in Dyninst.

We recently were able to run ~150 mutatees through one invocation of a Dyninst mutator.

– 11 – Dyninst 5.0

Register Optimizations Register Liveness Analysis

•Instrumentation point analyzed for which registers are live

•Basetramp tailored for specific instPoint– Only live registers need to be saved– Reduces overhead of instrumentation

•Implemented for GPR and FPR on Power•Implemented for GPR on x86_64

– 12 – Dyninst 5.0

Register Optimizations Register Usage Analysis

•Snippet analyzed for registers used•Basetramp tailored for snippets at that point

– Functions called from instrumentation checked for register usage

– Calls dealt with recursively

•Implemented for GPR and FPR on Power•Implemented for block FPR on x86 and

x86_64

– 13 – Dyninst 5.0

Register Optimization Naturally leads to minitramp in-lining

•Basetramp generated specifically for minitramp

•Snippet becomes one contiguous piece of code

•Enabled with BPatch::setMergeTramp(true);

One step closer to Dyninst as binary editor

– 14 – Dyninst 5.0

Register Optimizations This stuff works!

• Test written on Power architecture• Every basic block of mutatee instrumented

7.6

10.7

94.9

1.17

0 20 40 60 80 100

w/ RegisterLiveness

w/ RegisterUsage

Unoptimized

Uninstrumented

Time (s)

– 15 – Dyninst 5.0

Transactions More intelligent code generation requires a

more descriptive API•Code generated for multi-tramp and in-lining

highly sensitive to their associated snippets•Reduce unnecessary re-writes to mutatee

memory•Enabled from BPatch_process module

– BPatch_process::beginInsertionSet();– BPatch_process::finalizeInsertionSet(bool);

– 16 – Dyninst 5.0

Improved Testing Old Testsuite’s Monolithic Structure

•15 major test categories– Few developers know the meaning of all 15

•Poor distribution of subtests– 40 subtests in Test1– 1 subtest in Test15

•Difficult to add new subtests– Functionality goes untested

– 17 – Dyninst 5.0

Improved Testing New Testsuite

•Each subtest split into separate file– Easier to understand goal of test simply from

code– Easier to add new test to the system

•Compatible with old testsuite– Nightly runs logged in regression database along

with old testsuite

•Planned to replace old testsuite soon after 5.0 release

– 18 – Dyninst 5.0

Improved Testing parseThat

•Tool for testing Dyninst on arbitrary binaries

Two major goals•Allows for regular testing on production-

scale application binaries– Eg: GCC, Firefox, MySQL

•Allows for detailed bug reports from users

– 19 – Dyninst 5.0

X86_64 Support Introduces support for 64-bit mutatees One library to instrument them all

•32 and 64-bit mutatees both handled from same libdyninstAPI.so

•Mutatee word-length automatically recognized

Successfully running testsuite nightly

– 20 – Dyninst 5.0

Improved Windows Support Windows is important for Dyninst’s future

as a Security tool.

Now supporting Visual Studio 2003 More reliable stripped binary support Able to handle newer debugging

information formats.

Tested and works with the Microsoft Office suite.

– 21 – Dyninst 5.0

Miscellaneous Keeping up with the times

•GCC 4.x Support– Mutatees built from GCC 4.x– Building DyninstAPI with GCC 4.x

– 22 – Dyninst 5.0

Questions?

Matthew [email protected]

Ray [email protected]