42
TXBOX: BUILDING SECURE, EFFICIENT SANDBOXES WITH SYSTEM TRANSACTIONS Suman Jana The University of Texas at Austin Donald E. Porter Stony Brook University Vitaly Shmatikov The University of Texas at Austin IEEE Security & Privacy 2011

TxBox : Building Secure, Efficient Sandboxes with System Transactions

  • Upload
    reece

  • View
    36

  • Download
    0

Embed Size (px)

DESCRIPTION

Suman Jana The University of Texas at Austin Donald E. Porter Stony Brook University Vitaly Shmatikov The University of Texas at Austin IEEE Security & Privacy 2011. TxBox : Building Secure, Efficient Sandboxes with System Transactions. Outline. - PowerPoint PPT Presentation

Citation preview

Page 1: TxBox : Building Secure, Efficient Sandboxes with System Transactions

TXBOX: BUILDING SECURE, EFFICIENT SANDBOXES WITH SYSTEM TRANSACTIONS

Suman Jana The University of Texas at AustinDonald E. Porter Stony Brook UniversityVitaly Shmatikov The University of Texas at AustinIEEE Security & Privacy 2011

Page 2: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Outline

Introduction Building a Better Sandbox Related Work System Transactions Design and Implementation Evaluation Limitations Conclusions

Page 3: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Introduction

ACID: atomicity, consistency, isolation, and durability [wiki]

TxBox: Speculative execution of untrusted

application Uncircumventable enforcement of

system-call policies Automatic recovery from the effects of

malicious execution

Page 4: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Introduction

TxBox components: Grouping Rollback Access summary Access check

Based on TxOS [link] Linux with support for system

transactions Prevent race conditions

Page 5: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Building a Better Sandbox

TOCTTOU example

Softlink

Public file

Private file

1. Link to public file

2. Change to private file

System-call monitor must know all possible system-call sequences that lead to a violation and the monitor

must check every call !

Difficult and Slow !

Page 6: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Building a Better Sandbox

TxBox is immune to TOCTTOU attacks

Sandboxed processes run inside separate transactions

Whitelist / blacklist access-control policies

Page 7: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Building a Better Sandbox

Drawback of conventional sandbox Violation can be detected only after

multiple system calls, the damage may already have been done

TxBox: Transaction is aborted, and the system

automatically reverts

Page 8: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Building a Better Sandbox

Parallelism (Multi-core processors) Conventional sandbox or anti-virus:

Sandboxed program makes a system call, its execution must be paused

Split-personality malware TxBox:

Sandboxed program run with close-to-native performance while performing checks in parallel

Only a single copy is executed

Page 9: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Related Work

Speck, 2008 Multi-core processor Concurrently perform checks on an

instrumented copy on another core Cannot detect “split-personality”

malware Use Pin [link] Pin can be detected by malicious

program Suffer TOCTTOU attacks

Page 10: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Related Work

Page 11: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Related Work

Software Transactional Memory (STM) [wiki] Lock-based synchronization

STM-based systems Enforce application-specific security

policies Cannot span system calls

Page 12: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Related Work

Capsicum, 2010 Capability-based sandboxing system Help benign application to increase their

trustworthiness Native Client, 2009

Require the code to be recompiled to a restricted subset of the x86 ISA and also confines it using segmentation

Page 13: TxBox : Building Secure, Efficient Sandboxes with System Transactions

System Transactions

ACID properties for sequences of updates to system resources

If the transaction aborted before it finish, all intermediate updates are rolled back

All accesses to the system within a transaction are kept isolated and invisible to the rest of the system until the transaction commits

Page 14: TxBox : Building Secure, Efficient Sandboxes with System Transactions

System Transactions

TxOS: sys_xbegin – start transaction sys_xend – end transaction Maintain a transaction workset

Kernel objects (inodes, etc.) Each entry in workset contains a pointer to

the stable object, a pointer to the shadow copy

information: read-only / read-write Methods: commit / abort / lock / unlock /

release

Page 15: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Design and Implementation

Page 16: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Design and Implementation

TxBox policy BLACKLIST

Forbidden behavior WHITELIST

Required behavior DENY

Lists of system calls should be denied

Page 17: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Design and Implementation

BLACKLIST and WHITELIST primitives Regular expressions

System-call names and arguments, or objects Objects

(I) inodes Inode number Mode: read or write

(S) sockets Type: INET or UNIX Attributes: IP address (for INET socket) or names

(for UNIX datagram sockets)

Page 18: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Design and Implementation

Example (e)* means “match expression e any

number of times” Any deviation from a model-based

whitelist policy means that the sandboxed program is no longer executing the original code (code injection attacks)

Page 19: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Design and Implementation

Page 20: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Design and Implementation

DENY primitives DENY primitives is violated

Block calls and continue execution Run a sandboxed program without

letting it perform certain operations

Page 21: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Design and Implementation

Policy enforcement Character device driver

Send policy and pid to monitor Only communicate with root

Page 22: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Design and Implementation

System-call interposer TxBox call wrappers Symbolic link issue rename calls

Enforcer Force sandboxed

process to run in transaction mode

Page 23: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Design and Implementation

Policy decision engine Invoked when

sandboxed process attempts to perform a critical call

Page 24: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Design and Implementation

Implementation Based on Dazuko [wiki], by Avira

File Access Control Can also be based on LSM

xbegin -> beginTransaction kernel function beginTransaction checks if the current call

is xbegin Forced_transaction flag

Page 25: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Design and Implementation

Handling external I/O by sandboxed process Bidirectional communication cannot be

buffered until the end of a transaction Conventional system-call monitors

Call-by-call TxBox

If DENY primitive exists return error

Check WHITELIST and BLACKLIST Inspects the trace of the process Terminate and roll back

Page 26: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

Performance Test TxBox with Linux 2.6.22.6 Linux with Dazuko module Quad-core Intel X5355 2.66GHz 4GB RAM

Page 27: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

Micro-benchmarks Policy

BLACKLIST WREGEX *I:1234*

Worst system call for TxBox

Page 28: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

Application benchmarks

Page 29: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

Application benchmarks PostMark: a file system benchmark

Due to the fact that the transaction commit groups all writes and presents them to the I/O scheduler all at once, thus improving disk arm scheduling

Page 30: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

Scalability Opens 100 existing files Intel Core Due 2.00 GHz with 2GB RAM

Page 31: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

Original TxBoxfd =

open(“foo”, ..)

read(fd, ..)sockfd =

socket( .. )sendto =

(sockfd, .. )close(sockfd)close(fd)…

TX BEGINfd = open(“foo”, ..)read(fd, ..)sockfd = socket( .. )CHECK TX POLICY VIOLATIONNONTRANSACTIONAL sendto(sockfd, ..)close(sockfd)close(fd)…CHECK TX POLICY VIOLATIONTX END

Page 32: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

Non-transactional system call sendto / reccvfrom / connect / send /

recv / ioctl / read (from a socket) / write (to a socket)

IO-intensive applications wget System time increases by 30%-40%

Page 33: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

Overhead for trusted applications A single non-transactional system call

44% overhead Compile a non-transactional Linux kernel

Less than 2% overhead

Page 34: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

Creating files in a protected directory make install for vim BLACKLIST WREGEX *I:164564* 164564 -> /usr/local/bin/ Correctly roll back all effects on the

system

Page 35: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

Malicious MIME handler in a browser Tarhandler with lynx browser for MIME

type ‘application/x-tar’ Tarhandler reads file from /home/secret

and writes into /tmp/foo BLACKLIST WREGEX *I:183145* 183145 -> /home/secret

Page 36: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

Multimedia converter ffmpeg: a popular codec

Create output files in the /home/user1/ BLACKLIST WREGEX *I:181064*

181064 -> /home/user1

Page 37: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

Javascript engine Google V8 benchmark SpiderMonkey Javascript engine

Less than 5%

Page 38: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

On-access anti-virus scanning and parallelization of security checks ClamAV

Small file -> the overhead of transactional mechanism dominates

Page 39: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Evaluation

With 4 threads, TxBox is 2.3 times better than Linux+ Dazuko

Page 40: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Limitations

Kernel-based security monitor is vulnerable to kernel atttacks

Transactional semantics may change the behavior of sandboxed processes Transactional conflict Auto-retry

Transactional state is not shared Some anti-virus may need to observe

the execution of the program

Page 41: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Limitations

Colluding malware may evade security policies No sandboxing mechanism can reliably

prevent this Processes tha generate very large

worksets are killed DoS Attack

Page 42: TxBox : Building Secure, Efficient Sandboxes with System Transactions

Conclusions

Use system transactions Cannot be circumvented by

TOCTTOU Performance improvement (Multi-

core)