DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001Slide 1 Aegis Research Corporation Not for Public Release Intrusion Tolerance Using Masking, Redundancy

DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 1

Aegis Research Corporation

Not for Public Release

Intrusion Tolerance UsingMasking, Redundancy and Dispersion

DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001

Janet Lepanto

William Weinstein

The Charles Stark Draper Laboratory, Inc.

Aegis Research Corporation®

Aegis Research CorporationAegis Research Corporation




Overview

• Objectives and Assumptions

• System Design

– Overview

– Server Configuration Policy

– Configuration Monitoring

– Transaction Mediator

• Testing• Demonstration• Status and Near-Term Plans




Objectives and Assumptions

• Objectives

– Apply fault-tolerant design concepts to support intrusion tolerance

– Minimize loss of data confidentiality and integrity in the presence of a successful attack on one of the servers

– Tolerate attacks whose specific signatures are not known a priori

– Employ only a small set of trusted components to protect a large set of untrusted unmodified COTS servers and databases

– Employ redundancy for both intrusion tolerance and performance

• Assumptions

– Attacker desires stealth so transaction rates will be relatively low

– Attacks employing high transaction rates and recognizable signatures will be addressed by front-end firewalls and/or other intrusion detection mechanisms

– Exploitation of latent vulnerabilities will require more than a single transaction




System Design Overview




Technical Approach

• Mask fingerprints of gateway and origin servers so that an attacker cannot probe over network to determine

– OS of gateway, or origin servers

– Implementation of any origin server

• Distribute each client’s transactions among origin servers such that the client cannot predict which server will handle a transaction

• Periodically inspect each origin server for configuration anomalies that might indicate that attack transactions have occurred

– Reconfigure server to “clean” state if anomalies are detected

• Log transactions to back-end database so that data written by a compromised origin server can be reconstructed




Basic Architecture-Phase 2E

xter

nal

WA

N

ExternalFirewall

DataBase

TransactionMediator

Gateway

Sw

itch

ed I

P

Server(1)

Server(N)

Server(2)

Configuration Manager

Sw

itch

ed I

P

COTS

Trusted

Other




Configuration Management

Config Manager Origin Server

CM AgentConfiguration Management

Routines

Daemon, tailored to origin server OSMonitors server config info as directed by CM

Executes algorithms on origin server,(directory searches, hash calculations, etc.)

Reports status and transfers requested data to CM

Manages operation of CM AgentControls local IP switchFunctions as a client only

Manages Gateway operationDetermines servers in active set

Controls server utilization by updating “use/don’t use” status in Gateway

Origin Server Configuration

Templates

Structure of file system directory hierarchyExistence of specific files in specific directoriesHash integrity check of specific filesConfiguration data is dependent on server OS




Origin Server Configuration Policy

• Explicitly bind the http daemon and the CM agent to the external interface

– Don’t allow direct access from localhost to either one of the ports

– This prevents an attacker from hijacking the server port without killing the existing server daemon

• Run HTTP daemon as “nobody” (or the equivalent)

– Don’t run daemon as root

– Don’t allow “nobody” shell access

– This prevents an attacker from gaining immediate root privileges

• Run CM Agent daemon as root

– It needs most root privileges anyway, and giving it an account name would potentially make it easier to find

• Configure server-side scripts (cgi-bins) as read/execute only

• Don’t allow execution from the temp directory

• Don’t allow user-level processes to access RAM directly




Server -Side Scripting Policy

• All operational data is maintained on the back-end database server– All origin servers must remain functionally equivalent

– No client transaction data can persist on an origin server outside the duration of the transaction

– Because server-side scripts are constrained this way, they have no need to invoke root-level functions

• Server-side scripts– Will move data to and from the back-end servers via Transaction Monitor proxies

• File operations

• ODBC commands

– Access and user authentication• Via web server

• Via native database access management

• Back-end server administration– Done directly on servers

– Mediator can be configured to block administrative database operations




CM Agent – Data Flows

Input Buffer

MUTEX

CM Agent on Origin ServerCommands & reference data from CM

Configuration check results

Monitor & Control Flags and Reference Data

Tier 1 Monitor Thread

Tier 2a Monitor Thread

Tier 2b Monitor Thread

Tier 3 Monitor Thread

Shared Memory

Shared Memory

Shared Memory

Shared Memory

Shared Buffer

MUTEX

Shared Memory

Executive Thread

Config Man

Other Exec msgs to CM

Other info between Exec and Monitors

Output Buffer

MUTEXCM Interface

Thread

Heartbeat msg




Tier 1 Monitor

• Checks– Selected processes for proper name, status and PID value

• Use process config file to specify which process must be present (normal http daemon, agent daemon, and maybe others) and which other processes are allowed

• Use direct invocation of system calls for process ID, status checks and running privileges (this also validates that the server is running as “nobody”)

• Response to check fail– Specified in tier 1 config file (per each process) — responses include

• Leave server operating, invoke tier 2b check directly

• Take server off-line, invoke tier 3 check

• Rationale– Attempts to subvert the web server daemon or the monitoring agent processes will be

detected rapidly, before additional transactions occur that could allow further exploitation

Origin server activePeriod ≈ 1 second




Tier 2a Monitor

• Checks– Certain portions of the directory structure

• Server-side script files, html server files, certain system files

• All necessary files present, no extraneous files present

• Verify mod times on files

• Verify server configuration parameters

– Server-side scripts are read/executable only

– Temp directory scripts are read/write only

• Response to failure check– Take off-line and check logs to see if config changes were made by sys admin

• Rationale– Monitors for most likely points of exploitation and also picks up inadvertent

configuration changes by a system administrator

Origin server activePeriod ≈ 10 seconds




Tier 2b Monitor

• Checks– Hash of core files from Tier 2a

– Verify server configuration parameters

• Binding of http parent daemon to external interface

• User “nobody” has no shell access

• Response to failure check– Take off-line and do full tier 3 checks

• Rationale– Fairly frequent and thorough check of critical files for subtle changes

Origin server activePeriod ≈ 5-10 minutes




Tier 3 Monitor

• Checks– Full forensics, memory and file checkout

• Hash check all files

• Dump and check memory

• Response to failure check– Determine whether database might need recovery

– Scrub the platform

• Rationale– Detect whether a sophisticated attack has occurred and what damage

might have been done

Origin server inactivePeriod ≈ 1 hour or more




Back-end Operations

Back-end Databases

Origin Server

SQL Database(Brand Y)Web Server

Static Web Pages

Web Content

COTS

OASIS Developed

User Developed

Comms interface to Origin Servers & Config Man

Configuation Manager

Transaction Mediator

SQL Database(Brand X)

Back-end File Servers

File SystemBack-end

File ServerInterface

Scripts

ODBC Manager Interface

Proxy

Comms interface to Trans Monitor

App Server

InterfaceProxy

File Server

InterfaceProxy

Back-end App Servers

Applications

ODBCDriver X

ODBCDriver Y

ODBCManager

Monitor Proxy

TransactionLog

Database

Back-end App Server

Interface




Proxying ODBC

Back-end Databases

Origin Server

SQL DatabaseServer

(Brand X)

Transaction Log Database

ODBCDriver forBrand X

SQL DatabaseServer

(Brand Y)

ODBCManager

Comms interface to Trans Monitor

Web Server

ScriptsStatic Web Pages

Web Content

COTS

OASIS Developed

User Developed

CGI scripts produce SQL command strings to access back-end database—SQL flavor used depends on DB product accessed (same as if Origin Server accessed DB directly)

Supports traceability of ODBC commands to Origin servers and back-end database rollback

ODBC Manager interface proxy provides normal ODBC manager interface to CGI scripts

Comms interface to Origin Servers & Config Man

ODBC Manager Interface Proxy

Monitor Proxy

Config ManagerProvides back half of ODBC manager proxy—Manages operation of mediator

Transaction Mediator

ODBCDriver forBrand Y




Multi-Tier Database Connectivity Standards

• Standard SQL: International Standards Organization (ISO) and National Committee for Information Technology Standards (NCTIS) recognize a standard dialect of SQL known as SQL-92, published under similar and parallel standards ISO/IEC 9075-1992 and ANSI/NCTIS X3.135, respectively. Sub-levels of compliance exist within SQL-92 (Entry, Standard, Advanced).

• Call Level Interface: Call Level Interface (CLI) dates from 1995, and provides a standard set of API bindings for compiling languages (C, COBOL, Fortran, etc.) to guarantee portable execution of many SQL commands. A product of the Open Group, CLI is now an ISO standard extension of SQL (ISO/IEC 9075.3-1995).

• Microsoft’s Open Database Connectivity (ODBC) extends most of the features defined under CLI.




System Testing




Verification Testing Approach

• Subject the system to known scanning tools to determine if the mechanisms to thwart those scans are implemented properly

• Subject the system to known attacks to evaluate how it reacts to various types of attacks (e. g., measuring the relative time to success for an attack directly on server X vs. the same attack on server X operating in our OASIS architecture).

• Subject the system to modifications of known attacks developed to exploit knowledge of the architecture and operation of the system

• Setup system as a honeypot to observe character of real attacks and system response to those attacks




Verification Testing Status

• TCP/IP masking verification (Gateway and Origin Servers)– Fingerprinting attempts unsuccessful (nmap, queso, p0f, siphon)

– Fingerprinting attempts using ICMP errors unsuccessful (ICMP Port Unreachables)

• Proxy Scrubbing Verification– HTTP header cleanup successful

– Exception: Basic parameters of the HTTP request were cleaned, except for the HTTP Version (HTTP/1.0)

• Web page error documents– Intentional requests for non existent documents (404 Errors) allows

enumeration of origin servers

• Common Hacks Unsuccessful– Whisker (Times out due to the transaction dispersion algorithm)

– Win NT Unicode Exploit (Due to a patched server)




Verification Testing Plans

• Explore other ICMP techniques to discover the OS• Explore Reaction to Common Hacks

– Whisker (We are modifying to ignore time settings and server identification responses)

– WIN NT ISAPI Overflows• Index Server/Unicode Overflow Variant (Currently developed to stop

http process, will be modified to gain privileges)

• Print Server (Can overflow stack, being modified to initiate outbound trojan connections)

• Explore attacks against content– Content will be added to allow for a real world scenario test

– CGI scripts will be evaluated as a means of gaining access




Demonstration

• Transaction Dispersion• HTTP Whitening

Inte

rnet

GatewayOASIS Santa

Fe

Sw

itch

ed I

P

Solaris 2.7Apache 1.3.19

Windows 2000Apache 1.3.19

Open BSD 2.8Apache 1.3.19

Macintosh OS 9.1Personal Web Server




Program Status and Near-Term Plans

• Gateway w/redundant servers operating– Accessible over the Internet

– Independent sniffer monitoring traffic between Internet and Gateway

– Aegis conducting attack testing

– Plan to make system available as a target (to other OASIS projects)

• Server agents– Common structure is designed

– Coding underway for Unix and NT versions

• Transaction Monitor– Basic structure designed

– ODBC proxy mechanism being implemented




Task Schedule

1.1 Dev & Implmt Fingerprint Masking

1.2 Dev & Implmt Dynamic Assign

1.3 Dev & Implmt Config Evalution

1.4 Integration & Experimentation

1.5 Program Management

2.1 Dev & Implmt Transaction Mediator

2.2 Dev & Implmt Assess & Rollback


2.4/3.6 Dev Sync of Redundant Databases


3.1 Refine Fingerprint & Dynam Assign

3.2 Implement Sync & Voting

3.3 Refine Config Assessment



4.1 Refine Sync & Voting

4.2 Implmt Distribution of Servs & DBs



Task Name

7 10 1 4 7 10 1 4 7 10 1 4 7 10 12

CY 2000 CY 2001 CY 2002 CY 2003

Phase 1 – Basic Architecture

Phase 2 – Extended Architecture

Documents

DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001Slide 1 Aegis Research Corporation Not for Public Release Intrusion Tolerance Using Masking, Redundancy