Automated Tracing and Visualization of Software Security Structure and

Properties

Symposium on Visualization for Cyber Security 2012 (VizSec’12)Seattle, WA, USA

Oct. 15, 2012

 Wenbin Fang, Barton P. Miller, and James A. Kupsch

Computer Sciences DepartmentUniversity of Wisconsin-Madison

Motivation

Visualization: an intrinsic part of in-depth security assessment

• First Principles Vulnerability Assessment (FPVA)• Microsoft Threat Modeling

Diagrams as road map for later analysis• Key components and interaction• The privilege level of each component• Access to high-value resources

2

Example Diagrams From FPVA

3

Example Diagrams From FPVA

4

Example Diagrams From FPVA

5

Example Diagrams From FPVA

6

Diagram Creation Problems

Manual (time consuming) data collection • Collected from many sources• Potentially inaccurate

Manual diagram construction• Deferred until confident in data collection• Limits diagrams produced

Approach: Automate diagram construction7

Data Collection• Automatically collect trace data during runtime

Visualization• Construct diagrams/animation from trace data• Web-based interface

8

SecSTAR: Security SystemTracing, Analysis and Reporting

Data CollectionData Collection

InstrumentedBinary Code

Trace Data

VisualizationVisualization

Diagram Display Interface

Data Collection Overview

Goal: automate system data collection• Unmodified binaries• Follows control flows to other processes• Easy to extend to trace new security events

SecSTAR:• Uses self-propelled instrumentation• Simple code snippets determine what to

trace

9

Self-propelled Instrumentation

• Instrument unmodified binary code• No special preparation• Inject code snippet into a target process• Instrumentation follows control flow

• Within a process• Across thread boundaries• Across process and even host boundaries

10

Self-propelled Instrumentation

11

Application Process

• Injector: Process to inject shared library• Agent: Shared library

Injector processa.out

libc.so

libpthread.so

Agent.so

Payload Functions

Instrumentation Engine

12

void payload(SpPoint* pt){ if IsExit(pt) { trace(“exit” …) } else if IsConnect(pt) {

trace(“connect” …) } else if . . . // detect other events}

void main () {

pthread_create(foo …) …} void foo () {

connect(…)

exit(0)}

Host A Host BProcess P

Process Q

Agent.so

networkProcess R

Injector

Call

Call

Call

How it works

Detect system events• Process creation and destruction• Privilege level changes• Communication• Resource access

Query runtime info related to the current call• Arguments / Return value

Query Control Flow Graph (or CFG) structures• Functions / Basic blocks / Edges• Enables sophisticated code analysis

13

Payload Function

Visualization Overview

Goal: • Same-style same-quality diagrams as those

constructed by skilled analysts• Animate temporal data• Interactive interface

14

Data CollectionData Collection

InstrumentedBinary Code

Trace Data

VisualizationVisualization

Diagram Display Interface

Diagram, Animation and SecSTAR Interface

Demohttp://research.cs.wisc.edu/mist/projects/SecSTAR/

16

Case Study

• Using SecSTAR to produce FPVA-style diagrams for Condor

• Condor: high-throughput job scheduling system• Used worldwide• ~700,000 lines of code• 1000+ pages of documentation• Multiple processes, multiple hosts

17

Original FPVA vs SecSTAROriginal FPVA diagram construction

• Manual data collection from• Many processes and hosts• Documentation and code

• Correlated and distilled artifacts• Manual diagram creation• Months

SecSTAR• Automated data collection• Automated diagram construction• Hours, mostly to learn how to install and operate

Condor

18

Diagram comparison

19

SecSTAROriginal FPVA

Future Work

• Capture and visualize more events • Capture and visualize resources• Improve the web-based interface• Integrating with Microsoft Threat

Modeling

20

Summary

SecSTAR• Automated data collection • Automated diagram/animation construction

Case study• Diagram construction for Condor• Original FPVA vs SecSTAR

21

Questions?

http://www.cs.wisc.edu/mist/

22

Backup 1: Intra-process Propagation

23

a.outmain

8430:8431:8433:8444:8449:844b:844e:844f:

push %ebpmov %esp,%ebp...call printfmov %ebp,%espxor %eax,%eaxpop %ebpret

foo

callcalljmp

Patch1payload(foo)foo0x8405

Agent.so

callcalljmp

payload(printf)printf0x8449

Patch2

patchjmp

push %ebpmov %esp,%ebp...call foomov %ebp,%esppop %ebpret

83f0:83f1:83f3:8400:8405:8413:8414:

InjectActivatePropagate

jmp Patch1

jmp Patch2

Backup 2: Inter-process Propagation

24

Main procedure for inter-process propagation1. Detect the initiation of communication at the local site.

• connect, write, send …

2. Identify the remote process3. Inject the agent into the remote process4. Start following the flow of control in the remote site

void main () {

connect(…) recv(…)

}

void main () {

accept(…) send(…)

}

Agent.so Agent.so

inject

call payload()

Process A Process B