CSC 382/582: Computer SecuritySlide #1 CSC 382/582: Computer Security Integrity Management

CSC 382/582: Computer Security Slide #1

CSC 382/582: Computer Security

Integrity Management


Topics

1. Host Integrity

2. Anti-virus Software– Malware self-protection techniques.– Malware detection techniques.

3. Personal Firewalls

4. Host Intrusion Detection Systems

5. Host Intrusion Prevention Systems


Host Integrity

Ensuring a host adheres to a security policy.

C

AI

1. Misuse Detection and Prevention– Know a set of attacks against a host.– Attempt to detect and prevent these attacks.– Anti-virus, anti-spyware, personal firewall.

2. Policy-based Tools – Security policy describes good state of system.– Attempt to detect deviations from good state.– Host Intrusion Detection Systems (HIDS).


Misuse Detection

• Anti-virus tools most widely used host misuse detection software.– Signature database primary method of detecting

known attacks on host.

• Most anti-virus tools also offer– Intrusion Prevention: detect virus during d/l.– Recovery: eradicate detected viruses.

• Anti-virus software is often deployed on network proxy servers in addition to hosts.


Malware Self-ProtectionAnti-debugging

Detect/disable debuggers when used to analyze code.Attack anti-malware tools

Disable anti-malware tools upon infection.Kill processes or destroy/modify signatures.

API checksumsAvoid having UNIX/Win32 API calls in code.Store checksums of API names and search for match.

Code obfuscationUse unusual tricks and unused code to avoid dissassembly

and prevent quick analysis of purpose.Self-modifying code.


Self-Protection

CompressionCode looks almost random; size is smaller.

Use unusual executable packers to avoid analysis.

Data encryptionEncrypt strings, hostnames, IP addresses to avoid detection.

EmbeddingEmbed infection in one format inside a document inside an

archive file.

Scanners have to understand and have time to parse and decompress each file format.


Self-Protection

Entry-Point ObscuringChanging initial code or entry point easy to notice.

Alter program code to gain control randomly.

Host morphingAlter host file during infection to prevent removal.


Self-Protection: Encryption

Encrypt all code except small decryptor.– Note that copy protected files will have similar

decryptors to prevent analysis too.– Often uses multiple decryptors.– Change encryption key dynamically.

Random Decryption Algorithm (RDA)– Choose random key for encryption.– Brute force search for key to decrypt.– Slows VMs/debuggers used for analysis.


Self-Protection: Polymorphism

Alter malware code with each infection.– Cannot be detected by signature scanning.– May alter decryptor only or entire code.– Insert junk instructions that do nothing.– Fragment and rearrange order of code.– Alternate sets of instructions for the same task.

• Ex: SUB -1 instead of ADD 1

– Randomize names in macro viruses.


Case Study: Zmist

EPO, encrypted, polymorphic virus.

Code integrationDecompiles PE files to smallest elements.

Inserts virus randomly into existing code.

Rebuilds executable.

Polymorphic decryptorInserted as random fragments linked by JMPs.

Randomizes self with ETG engine.


Virus Detection

Signature-based – Look for known patterns in malicious code.– Defeated by polymorphic viruses.

Smart scanning– Skips junk instructions inserted by poly engines.– Skips whitespace/case changes in macro viruses.

Decryption– Brute-forces simple XOR-based encryption.– Checks decrypted text against small virus sig to

decide whether has plaintext or not.


Virus Detection

Code Emulation– Execute potential malware on VM.– Scan VM memory after certain # iterations.– Watch instructions for decryptor profile.

Code Optimization.– Optimize away junk instructions and odd

techniques used by polymorphic viruses.


Virus Detection

Heuristics– Code execution starts in last section.– Suspicious code redirection.– Suspicious section ACLs or size.– Suspicious library routine imports.– Hard-coded pointers into OS kernel.

Neural Network Heuristics– IBM researchers trained neural net to recognize

difficult polymorphic viruses.– Released in Symantec antivirus.


Limits of Malware Detection

• Assume you have a perfect malware detector D(p) that takes a program p as input and returns True or False.

• Create a program P(q) that incorporates your malware detector D:if D(q):

Do nothingelse

Become malware

• What would D report if given P to analyze?


Personal Firewall

• Firewall configured to protect single host.– Used on servers, desktops, and laptops.

• Why use a personal firewall?– Configuration can closely match single host’s

needs without considering entire network.– Can protect on a per-application basis.– Can protect mobile hosts when outside the

organization’s network firewall.


Host Intrusion Detection Systems

Monitors host state for signs of intrusion:1. Files

• File metadata changes (access time, perms)

• File data changes (checksums)

2. Configuration • Log file entries

• User accounts and groups

3. Runtime• Logins and logouts

• Running processes

• Open network connections

• Kernel modules and status


Why HIDS?

• October 25, 2000– Microsoft detects passwords being mailed out

of company to e-mail address in Russia.– Electronic logs show source code downloaded.

• How did it start?– Employee received e-mail carrying Qaz trojan.– Qaz copied itself to Notepad.exe, moved

Notepad to Note.com.– Qaz spread itself across network, d/led tools.– Attacker used tools to acquire passwords.


Advantages of HIDS

1. HIDS can associate data with specific users, while NIDS cannot do this.

2. HIDS has access to data that’s sent to the host in a network encrypted form (SSL, VPN.)

3. HIDS is immune to NIDS evasion techniques. NIDS doesn’t know how host stack interprets packets, but HIDS looks at data after stack has processed it.


HIDS Types

1. File Integrity Checkers– Tripwire, AIDE, anti-virus software

2. Log Watchers– logwatch, swatch

3. Network monitors– portsentry, BlackICE

4. Host Integrity Monitors– Osiris, Samhain


File Integrity Checkers

1. Perform baseline scan of filesystem.– Metadata: ownership, permissions, times– Cryptographic checksums of contents

2. Periodically scan filesystem– Compare current state to baseline state.– Notify admin if changes discovered.


File Integrity Checkers

Configuration required to avoid false +’s– Ignore temporary files in /tmp and elsewhere.– Ignore log file checksums, but permissions and

ownerships are important and size should not decrease.

– Update checksums when binaries or libraries updated.


HIDS Architecture

A HIDS consits of three components:– Agent runs on host and gathers data.

– Director periodically polls hosts and aggregates data• Sends requests to agents to scan hosts.

• Receives data from agents.

• Decides on whether to act on data.

– Notifier acts on director results.• May simply notify security officer.

• May reconfigure agents or director.

• May activate response mechanism.


Agent• Scans host and transmits data.• Attackers target agent to avoid detection

– Kill agent process.– Replace agent with their own code.– Install rootkit.

• Protection against subversion– Self integrity check.– Privilege separation.– Encrypted communication with director.– Configuration data and baselines stored on director.– Scan data transmitted to director host only.


Director

Director Functionality1. Polls agents to request scan data.

2. Sends configuration data to agents.

3. Receives scan data from agents.

4. Stores scans in database.

5. Compares scans with baselines.

6. Updates baselines if Administrator requests.

Protecting the Director– Use a dedicated host with limited access.

– Run an agent on the director host to monitor itself.


Example HIDS: Tripwire


Running a HIDS

• When should I collect baselines?– Ideally before the host is connected to network.– If later, knowngoods.org maintains checksums

of common Linux distributions.

• How often should I poll hosts?– Depends on performance goals.

• How often should I monitor alerts?– Serious alerts should automatically notify you.– Read logs daily.


Host Intrusion Prevention Systems

• Attacks usually cause programs to initiate new behaviors:– New network sockets opened.– New files modified, etc.

• Create a system call model of a program.– Compiler extracts call graph model of program.– HIPS monitors program during execution,

checking to see if execution matches model.


Example: System Call Models


Key Points

1. Integrity Management Approaches– Misuse detection and prevention.– Policy-based management.

2. Detection Issues– Problems: encryption, polymorphism– Solutions: compiler-based, VM simulations

3. Host Integrity Management Systems– System Types: HIDS, HIPS– Monitor Files, Configuration, Runtime data.– HIDS provides local view with greater detail than

NIDS.


References1. Matt Bishop, Computer Security: Art and Science, Addison-Wesley, 2003.2. Ted Bridis and Rebecca Brucman, “Microsoft hacked! Code stolen?”,

http://news.zdnet.com/2100-9595_22-525083.html, October 26, 2000.3. Simson Garfinkel, Gene Spafford, and Alan Schartz, Practical UNIX and

Internet Security, 3/e, O’Reilly & Associates, 2003.4. Lap Chung Lam, Wei Li, and Tzi-cker Chiueh, “Accurate and Automated

System Call Policy-Based Intrusion Prevention,” Proceedings of the International Conference on Dependable Systems and Networks (DSN'06), 2006.

5. Stephen Northcutt, Lenny Zeltser, Scott Winters, Karen Kent and Ronald Ritchey, Inside Network Perimeter Security, Second Edition, Sams Publishing, 2005.

6. Ed Skoudis and Lenny Zeltser, Malware: Fighting Malicious Code, Prentice Hall, 2003.

7. Ed Skoudis, Counter Hack Reloaded 2/e, Prentice Hall, 2006.8. Peter Szor, The Art of Computer Virus Research and Defense, Addison-

Wesley, 2005.9. Brian Wotring, Host Integrity Monitoring Using Osiris and Samhain,

Syngress, 2005.

http://news.zdnet.com/2100-9595_22-525083.html

Documents

CSC 382/582: Computer SecuritySlide #1 CSC 382/582: Computer Security Integrity Management