Upload
gaye
View
26
Download
0
Embed Size (px)
DESCRIPTION
Playing Inside the Blackbox: Using Dynamic Instrumentation to Create Security Holes. Barton P. Miller [email protected] Computer Sciences Department University of Wisconsin Madison, Wisconsin 53705 USA. Overview. 1. How to easily do dangerous and malicious things to a running program. - PowerPoint PPT Presentation
Citation preview
DynInst Security© 2001 Barton P. Miller December 2001
Playing Inside the Blackbox:Using Dynamic Instrumentation to Create
Security Holes
Barton P. [email protected]
Computer Sciences DepartmentUniversity of Wisconsin
Madison, Wisconsin 53705USA
– 2 – DynInst Security© 2001 Barton P. Miller
Overview
1. How to easily do dangerous and malicious things to a running program.
2. How to detect when someone does something evil to your program.
– 3 – DynInst Security© 2001 Barton P. Miller
A New ViewRunning programs are objects to be easily
manipulated. Kinds of manipulations might include:
Instrumentation
Optimization
Control
– 4 – DynInst Security© 2001 Barton P. Miller
The Vehicle:The DynInst API
A machine-independent library for machine level code patching.
Eases the task of building new tools.
Provides the basic abstractions to patch code on-the-fly
– 5 – DynInst Security© 2001 Barton P. Miller
Dynamic InstrumentationDoes not require recompiling or relinking
• Saves time: compile and link times are significant in real systems.
• Can instrument without the source code (e.g., proprietary libraries).
• Can instrument without linking (relinking is not always possible.
Instrument optimized code.
– 6 – DynInst Security© 2001 Barton P. Miller
Dynamic Instrumentation (con’d)Only instrument what you need, when you
need• No hidden cost of latent instrumentation.• Enables “one pass” tools.
Can instrument running programs (such as Web or database servers)• Production systems.• Embedded systems.• Systems with complex start-up procedures.
– 7 – DynInst Security© 2001 Barton P. Miller
The Basic MechanismApplicationProgram
Function foo
Trampoline
Pre-InstrumentationRelocatedInstruction
Post-Instrumentation
– 8 – DynInst Security© 2001 Barton P. Miller
The DynInst InterfaceMachine independent representationObject-based interface to build Abstract
Syntax Trees (AST’s)Write-once, instrument-many (portable)Hides most of the complexity in the API
• Process Hijacker: only 700 lines of user code!
• MPI tracer: 250 lines
– 9 – DynInst Security© 2001 Barton P. Miller
Basic DynInst OperationsProcess control:
• Attach/create process• Monitor process status changes• Callbacks for fork/exec/exit
Image (executable program) routines:• Find procedures/modules/variables• Call graph (parent/child) queries• Intra-procedural control-flow graph
– 10 – DynInst Security© 2001 Barton P. Miller
Basic DynInst OperationsInferior (application processor) operations:
• Malloc/free– Allocate heap space in application process
• Inferior RPC– Asynchronously execute a function in the
application.• Load module
– Cause a new .so/.dll to be loaded into the application.
– 11 – DynInst Security© 2001 Barton P. Miller
Basic DynInst OperationsInferior operations (continued):
• Remove Function Call– Disable an existing function call in the
application• Replace Function Call
– Redirect a function call to a new function• Replace Function
– Redirect all calls (current and future) to a function to a new function.
– 12 – DynInst Security© 2001 Barton P. Miller
Basic DynInst OperationsBuilding AST code sequences:
• Control structures: if and goto• Arithmetic and Boolean expressions• Get PID/TID operations• Read/write registers and global variables• Read/write parameters and return value• Function call
– 13 – DynInst Security© 2001 Barton P. Miller
Security Applications of DynInst
Lots of tool applications of Dyninst by lots of groups. Here are two security-oriented ones:
License server bypassing
Condor security attacks
– 14 – DynInst Security© 2001 Barton P. Miller
License Server Attack: The Bypass
Program License DataNetwork
License
Server
Normal: licensed program runs after communicates with license server.
Program License DataNetwork
License
Server
Undesired: licensed program refuses to run if license server does not respond.
– 15 – DynInst Security© 2001 Barton P. Miller
Example: Adobe FrameMakerTwo-step license verification:
• retrieve license data from server [once]• check license data for correctness [often]
In practice:• allow FM to time-out waiting for server• allow FM to attempt to go into “demo” mode• switch FM back to normal mode• insure that future license checks always
succeed
– 16 – DynInst Security© 2001 Barton P. Miller
StrategiesComplete reverse engineering:
• not an option– legal problems– complexity (FrameMaker is a 7 MB binary!)
Focus on certain characteristics:• I/O (network sockets) traffic• execution trace
– 17 – DynInst Security© 2001 Barton P. Miller
ToolsHigh-level language translators:
• Dyner: interactive, interpreted C subset• Jdyninst: Java to DynInst compiler
Bypasser: an interactive call graph browser• Search and walk application call graph• Resolves function pointers at runtime• Call follow caller or callee paths• Can generate call trace
– 18 – DynInst Security© 2001 Barton P. Miller
UseDetermining where to apply changes:
• get trace for a successful run• get trace for a (forced-)failure run• compare to find differences• repeat as needed
– 19 – DynInst Security© 2001 Barton P. Miller
DetailsFM calls NlOpenlicenses on start up
• Contacts license server and caches credential if successful
At end of main, FM calls NluiCheckLicense• If credential is not present, call ChangeProductToDemo (cannot save files)
Frequently, during operation, FM will check for cached credentials.
– 20 – DynInst Security© 2001 Barton P. Miller
DetailsFM calls NlOpenlicenses on start up
• Contacts license server and caches credential if successful
• Allow this to fail.At end of main, FM calls NluiCheckLicense
• If credential is not present, call ChangeProductToDemo (cannot save files)
Frequently, during operation, FM will check for cached credentials.
– 21 – DynInst Security© 2001 Barton P. Miller
DetailsFM calls NlOpenlicenses on start up
• Contacts license server and caches credential if successful
• Allow this to fail.At end of main, FM calls NluiCheckLicense
• If credential is not present, call ChangeProductToDemo (cannot save files)
• Delete the call to ChangeProductToDemo.Frequently, during operation, FM will check
for cached credentials.
– 22 – DynInst Security© 2001 Barton P. Miller
DetailsFM calls NlOpenlicenses on start up
• Contacts license server and caches credential if successful
• Allow this to fail.At end of main, FM calls NluiCheckLicense
• If credential is not present, call ChangeProductToDemo (cannot save files)
• Delete the call to ChangeProductToDemo.Frequently, during operation, FM will check
for cached credentials.• Change this call to always return “true”.
– 23 – DynInst Security© 2001 Barton P. Miller
Condor Attack: Lurking JobsCondor schedules jobs on idle workstationsIn a normal mode, jobs run as a common,
low-privilege user ID: “nobody”.This common user ID provides an
opportunity for an evil lurking process to ambush subsequent jobs (from other users):
– 24 – DynInst Security© 2001 Barton P. Miller
Condor Job Structure
Submitting Host
Shadow Process
Execution Host
User Jobsystem calls
– 25 – DynInst Security© 2001 Barton P. Miller
Condor Job Structure
Submitting Host
Shadow Process
Execution Host
EvilUser Job
system calls
LurkerProcess
fork
– 26 – DynInst Security© 2001 Barton P. Miller
Condor Job Structure
Submitting Host Execution Host
LurkerProcess
– 27 – DynInst Security© 2001 Barton P. Miller
Condor Job Structure
Submitting Host
Shadow Process
Execution Host
InnocentUser Job
system calls
LurkerProcess
– 28 – DynInst Security© 2001 Barton P. Miller
Condor Job Structure
Submitting Host
Shadow Process
Execution Host
InnocentUser Job
system calls
LurkerProcess
attach
– 29 – DynInst Security© 2001 Barton P. Miller
Condor Job Structure
Submitting Host
Shadow Process
Execution Host
InnocentUser Job
system calls
Control remotesystem calls
LurkerProcess
rm -rf *
– 31 – DynInst Security© 2001 Barton P. Miller
– 32 – DynInst Security© 2001 Barton P. Miller
Can We Trust a Remote Job?The threats:
1. Cause the job to make improper remote system calls.
2. Cause the job to calculate an incorrect answer.3. Steal data from the remote job.
Threat protection strategies:• File sand-boxing (#1)• System call sand-boxing (#1)• Obscure and encode binary (#1)• Replicate remote job (#2)
– 33 – DynInst Security© 2001 Barton P. Miller
Sand-Boxing
Shadow process selectively rejects system calls:• Restrict access to specific files or directories• Disallow certain system calls• Disallow certain system call parameter values
Submitting Host
Shadow Process
Execution Host
User Jobsystem calls
– 34 – DynInst Security© 2001 Barton P. Miller
Obscuring the Executable
User Job
ModifiedUser JobChecking Shadow
Modifier/Obscurer
– 35 – DynInst Security© 2001 Barton P. Miller
Obscuring the Executable
Goal:Even if an intruder can see, examine, and fully
control the remote job, no harm can come to the local machine.
– 36 – DynInst Security© 2001 Barton P. Miller
Obscuring the ExecutableModify the executable:
• Replace each system call site with call to a unique random function nameThis makes understanding the executable harder and
makes checking the system call stream easier.• Delete any constant parameters
Values like file names or descriptor numbers are hidden.• Pad parameter other lists so all calls have equal
number• Permute the order of parameters• Add dummy system calls throughout code
– 37 – DynInst Security© 2001 Barton P. Miller
Obscuring the ExecutableGenerate a Checking Shadow:
• Build a FSM that reflects the inter-procedural control-flow of the job.Note that this FSM has increased accuracy, since each
call site has a unique name• Map random function names back to their
actual system calls.• Insert parameter values and present in correct
order.• Ignore dummy calls.
– 38 – DynInst Security© 2001 Barton P. Miller
Obscuring the ExecutableSome notes of interest:
• All hiding and replacement is done before the job is submitted to Condor.No knowledge of the encryption or keys is contained in
the remote job.• All replacing is done in the Checking Shadow,
which only resides on your local machine.• FSM accuracy and computational complexity is
simplified by the function renaming.Our early results look good. A tech report will
be available by the end of November.
– 39 – DynInst Security© 2001 Barton P. Miller
How to Get a Copy of DynInst:Release 2.3 (release 3.0 imminent)
• Free for research use.• Runs on Solaris (SPARC & x86), Windows
NT, AIX/SP2, Linux (x86), Irix (MIPS),Tru64 Unix (Alpha).
http://www.paradyn.orghttp://[email protected]