Upload
duongtram
View
258
Download
5
Embed Size (px)
Citation preview
Server Forensics: Linux
Part II.A. Techniques and Tools: Computer Forensics
CSF: Forensics Cyber-Security Fall 2015
Nuno Santos
Summary
2015/16 CSF - Nuno Santos 2
} Linux forensics
} Overview of Linux (Unix) systems
} System subversion with rootkits
} Rootkit detection techniques
Operating systems we will be focusing on
2015/16 CSF - Nuno Santos 3
} Desktop platforms: Windows
} Server platforms: Linux
} Mobile platforms: Android
Today!
Previous class
Our main focus when analyzing a server’s OS
2015/16 CSF - Nuno Santos 4
} On desktops, we were mostly concerned about evidence left by local users’ activities
} On servers, we will look for evidence left by external agents that resulted in security violations to the OS
} Such external agents typically consist of stealthy malware called rootkits
attack
rootkit
Overview of Linux systems
2015/16 CSF - Nuno Santos 5
Linux belongs to the big family of Unix systems
2015/16 CSF - Nuno Santos 6
Unix systems’ bootstrap sequence
2015/16 CSF - Nuno Santos 7
} At a high-level, Unix systems have same boot sequence
Unix systems’ architecture
2015/16 CSF - Nuno Santos 8
} Unix systems also have same arch, at a high-level } The kernel can be extended with loadable modules
System subversion with rootkits
2015/16 CSF - Nuno Santos 9
What’s wrong with this picture?
2015/16 CSF - Nuno Santos 10
[root@dobro bin]# ls -‐a . dd igawk nisdomainname tar .. df ipcalc pgawk tcsh .. dmesg kbd_mode ping touch alsaunmute dnsdomainname kill ping6 tracepath arch doexec ksh ps tracepath6 ash domainname link pwd traceroute ash.static dumpkeys ln red traceroute6 aumix-‐minimal echo loadkeys rm true awk ed login rmdir umount
Let’s take a closer look
2015/16 CSF - Nuno Santos 11
How can there be two “..” directories?...
[root@dobro]# ls -‐a . dd igawk nisdomainname tar .. df ipcalc pgawk tcsh .. dmesg kbd_mode ping touch
How did this happen?
2015/16 CSF - Nuno Santos 12
} This is actually:
mkdir <dot><dot><backslash><space><enter>
} It creates a directory named “dot-dot-space”
[root@dobro bin]# mkdir ..\
What’s in this “mystery” directory?
2015/16 CSF - Nuno Santos 13
Here’s a simple trick to hide malicious files in plain sight
[root@dobro bin]# cd ..\ [root@dobro .. ]# ls -‐l total 24 -‐rw-‐r-‐-‐r-‐-‐ 1 root root 0 Dec 15 12:19 rootkit_file_01 -‐rw-‐r-‐-‐r-‐-‐ 1 root root 0 Dec 15 12:19 rootkit_file_02 -‐rw-‐r-‐-‐r-‐-‐ 1 root root 0 Dec 15 12:19 rootkit_file_03 -‐rw-‐r-‐-‐r-‐-‐ 1 root root 0 Dec 15 12:19 rootkit_file_04 -‐rw-‐r-‐-‐r-‐-‐ 1 root root 0 Dec 15 12:19 rootkit_file_05 -‐rw-‐r-‐-‐r-‐-‐ 1 root root 0 Dec 15 12:19 rootkit_file_06
Rootkits
2015/16 CSF - Nuno Santos 14
} The behavior of the operating system can be affected by the presence of rootkits
} Rootkits are a category of malware which has the ability to hide itself and cover up traces of activities
Sample scenario
2015/16 CSF - Nuno Santos 15
} A rootkit runs a password logger on a compromised system
Compromised system
Password Logger
Email with sniffed passwords encoded
using some text-based steganography
techniques
Another example
2015/16 CSF - Nuno Santos 16
} A rootkit installs a backdoor on the compromised system
Compromised system
Backdoor communication with backdoor (covert channel
in HTTP or DNS)
Shell
Rootkit goals
2015/16 CSF - Nuno Santos 17
1. Remove evidence of original attack and activity that led to rootkit installation
2. Hide future attacker activity (files, net connections, processes) and prevent it from being logged
3. Enable future access to system by attacker
4. Install tools to widen scope of penetration
5. Secure system so other attackers can’t take control of system from original attacker
Rootkits include several attacker tools
2015/16 CSF - Nuno Santos 18
} Attack tools installed after an intruder gained access: } Backdoor programs } Network sniffers } Log cleaners } File / process / user hiding tools } etc.
Rootkit tools: Backdoor programs
2015/16 CSF - Nuno Santos 19
} Backdoor is an unauthorized way of gaining access to a program, online service or an entire computer system } Let attackers log in to the hacked system w/o using an exploit again
Examples Description
Login Backdoor Modify login.c to look backdoor password before stored password
Telnetd Backdoor Trojaned the “in.telnetd” to allow attacker gain access with backdoor password
Services Backdoor Replacing and manipulate services like “ftp”, “rlogin”, even “inetd” as backdoor to gain access
Cronjob backdoor Backdoor could also be added in “crontjob” to run on specific time for example at 12 midnight to 1 am
Library backdoors Shared libraries can be backdoor to do malicious activity including giving a root or administrator access
Kernel backdoors This backdoor is basically exploiting the kernel
Rootkit tools: Sniffers and wipers
2015/16 CSF - Nuno Santos 20
} Packet sniffers } Packet Sniffer is a program and/or device that monitor data
traveling over a network, TCP/IP or other network protocol } Used to listen or to steal valuable information off a network; many
services such as “ftp” and “telnet” transfer their password in plain text and it is easily capture by sniffer
} Log-wiping utilities } Log file are the lists actions that have occurred, e.g., in UNIX, wtmp
logs time and date user log in into the system } Log file enable admins to monitor, review system performance and
detect any suspicious activities } Deleting intrusion records helps prevent detection of the intrusion
Rootkit tools: Miscellaneous attacker tools
2015/16 CSF - Nuno Santos 21
} DDOS program } To turn the compromised server into a DDOS client such as, trinoo
} IRC program } Connects to some remote server waiting for the attacker to issue a
command (e.g., to trigger a distributed denial of service attack)
} System patch } Attacker may patch the system after successful attack; this will
prevent other attacker to gain access into the system again
} Password cracker } Vulnerability scanners } Hiding utilities
} Utilities to conceal the rootkit files on compromised system
Rootkit types
2015/16 CSF - Nuno Santos 22
} Command-level rootkits } Replace user-level programs
} Library-level rootkits } Replace system libraries
} Kernel-level rootkits } Modify the kernel itself
Command-level subversion
2015/16 CSF - Nuno Santos 23
} Command-level rootkits hide the presence of malware by making changes to system commands } Based on principle: To suppress bad news, silence the messenger
} Table shows typical command-level rootkit modifications
Evolution of command-level rootkits
2015/16 CSF - Nuno Santos 24
} First rootkits came with network sniffers to collect user names and passwords, e.g., the esniff program
} Later versions came with remotely controlled agents for distributed denial-of-service attacks, e.g., T0rn
Command-level rootkit evasion
2015/16 CSF - Nuno Santos 25
} To evade detection, early rootkits not only replaced system utility sw but also erased records in system logs } Some rootkits even gave modified system utilities the same file
time stamps and cyclic redundancy check (CRC) values as the original files
} Later command-level rootkits simply install modified programs that hide the presence of malware
Installation concealment
2015/16 CSF - Nuno Santos 26
} Use a subdirectory of a busy system directory like /dev, /etc, /lib, or /usr/lib
} Use dot files, which aren’t in ls output
} Use spaces to make filenames look like expected dot files: “. “ and “.. “
} Use filenames that system might use } /dev/hdd (if no 4th IDE disk exists) } /usr/lib/libX.a (libX11 is real Sun X-Windows)
} Delete rootkit install directory once installation is complete
Rootkit types
2015/16 CSF - Nuno Santos 27
} Command-level rootkits
} Library-level rootkits
} Kernel-level rootkits
Library-level subversion
2015/16 CSF - Nuno Santos 28
} Instead of replacing system utilities, rootkits can hide their existence by making changes at the next level down in the system architecture: the system run-time library
} A good example is redirecting the open() and stat() calls } The purpose of these modifications is to fool file-integrity-checking
software that examines executable file contents and attributes } By redirecting the open() and stat() calls to the original file, the
rootkit makes it appear as if the file is still intact } However, execve() executes the subverted file
Example of library-level subversion
2015/16 CSF - Nuno Santos 29
} Redirect specific open() system calls
real_syscall3() is a macro (not entirely shown) that modifies the standard _syscall13() macro real_syscall3 is defines our real_open() function that invokes Sys_open system call
Rootkit types
2015/16 CSF - Nuno Santos 30
} Command-level rootkits
} Library-level rootkits
} Kernel-level rootkits
Kernel-level subversion
2015/16 CSF - Nuno Santos 31
} Just like user-level rootkits, kernel-level rootkits are installed after a system’s security has been breached
} Kernel-level rootkits compromise the kernel } Kernel runs in supervisor processor mode } Thus, the rootkit gains complete control over the machine
} Advantage: stealth, e.g., } Runtime integrity checkers cannot see rootkit changes } All programs in the system can be affected by the rootkit } Open backdoors/sniff network without running processes
Kernel rootkit operation
2015/16 CSF - Nuno Santos 32
} The purpose of many kernel rootkits is to hide malware processes, files, and network ports (& the rootkit itself)
} There are to sides to information hiding
} Output: the kernel must censor the output from system calls that produce a list of processes, files, network ports, etc.
} Input: any attempts to manipulate a hidden process, file, network port, and so on must fail as if the object did not exist } May also redirect system calls to subvert operation of software that
checks integrity of executable files
Methods to inject rootkit code into a kernel
2015/16 CSF - Nuno Santos 33
1. Loading kernel module into a running kernel } Uses official LKM interface, hence it’s easier to use } Hide module names from external (/proc/ksyms) & internal tables } Might also intercept syscalls that report on status of kernel modules
2. Injecting code into the memory of a running kernel that has no support for module loading
} Involves writing new code to unused kernel memory via /dev/kmem and activating the new code by redirecting, e.g., a system call
3. Injecting code into the kernel file or a kernel module file } These changes are persistent across boot, but require that the
system is rebooted to activate the subverted code
Early kernel rootkit architecture
2015/16 CSF - Nuno Santos 34
} Based on system-call interposition: Early kernel rootkits subvert syscalls close to the process-kernel boundary
Rootkit interposition code
2015/16 CSF - Nuno Santos 35
} To prevent access to a hidden file, process, and so on, rootkits redirect specific system calls to wrapper code
} To prevent rootkit disclosure, syscalls that produce lists of files, etc., are intercepted to suppress info to be hidden
System-call interposition: good and bad
2015/16 CSF - Nuno Santos 36
} Advantage: The code is easy to understand } The change is made at a point that is close to the user of
those system calls
} Disadvantage: Many system calls need to be intercepted } E.g., to hide the existence of a file, need to intercept all
system calls that have file name arguments, about 50 syscalls
} Alternative: subvert the kernel at a level that is closer to the information being hidden
Later kernel rootkit architectures
2015/16 CSF - Nuno Santos 37
} Based on file system interposition: Rootkits based on interposition at object interfaces subvert several FS methods
Hiding file names
2015/16 CSF - Nuno Santos 38
} Of all the operations provided by the VFS interface the lookup() operation is of particular interest } lookup() translates the pathname into underlying FS object } Redirect can hide a file from all syscalls that access a file by name
} Doesn’t work for syscalls that create a new directory entry } E.g., link(), mkdir(), socket(), etc. } Unless the malware does not handle this, operation will fail
Rootkit detection techniques
2015/16 CSF - Nuno Santos 39
Detection techniques for each rootkit type
2015/16 CSF - Nuno Santos 40
} Command-level rootkits
} Library-level rootkits
} Kernel-level rootkits
Approaches to detecting command-level rootkits
2015/16 CSF - Nuno Santos 41
1. If a rootkit listens for connections, the network port will be visible to an external network port scanner
2. Some tools can reveal the names of all directory entries, including hidden or deleted files
3. Corrupted versions of ps and similar hide malware processes, but these can still be found using, e.g., the /proc file system
4. Deleted login/logout records in the wtmp file leave behind holes that can be detected using an appropriate tool
5. Ifconfig might report that a network interface is not in sniffer mode, but we can query the kernel for the interface status
6. CRC checksums reported by compromised cksum, can be detected using MD5 or SHA1
7. When examining a low-level copy of the file system on a trusted machine, all hidden files and modifications will be visible
Example of exposing a command-level rootkit1/7
2015/16 CSF - Nuno Santos 42
} We examine two utilities of the T0rn rootkit for Linux } This rootkit was in widespread use in 2001
} First, search the /bin/ls executable file using strings and grep
} The file name /usr/src/.puta/.1file looks very suspicious
Example of exposing a command-level rootkit2/7
2015/16 CSF - Nuno Santos 43
} If we try to list the /usr/src/.puta directory, the ls command hides the name, as we would expect:
} However, the directory name still shows up when we use the echo command, together with the .* wildcard expansion feature that is built into the command shell
Example of exposing a command-level rootkit3/7
2015/16 CSF - Nuno Santos 44
} File .puta/.1file is a rootkit configuration file } Has list of file and directory names that must remain hidden } Contain the malware program files, config files, and data files
Example of exposing a command-level rootkit4/7
2015/16 CSF - Nuno Santos 45
} To detect modified process status check against /proc } Table shows that ps command is hiding with ID 153
Example of exposing a command-level rootkit5/7
2015/16 CSF - Nuno Santos 46
} The system utilities that were replaced by the rootkit do a good job of hiding process 153
} Not only it is censored by process status tools like ps, but also it does not show up with network status tools such as netstat
} However, it does not replace the lsof command, which can help reveal the purpose of process 153
Example of exposing a command-level rootkit6/7
2015/16 CSF - Nuno Santos 47
} File name /usr/sbin/nscd suggests that it is a system program, but comparison with uncompromised systems shows this program exists only in later Linux versions } nscd: name service cache daemon
Example of exposing a command-level rootkit7/7
2015/16 CSF - Nuno Santos 48
} Connecting with telnet to TCP port 47017 on the local machine confirms it is a backdoor process } In this case we are welcomed by the opening banner of what
appears to be an SSH server
} Find more about the T0rn rootkit at: https://www.sans.org/security-resources/malwarefaq/t0rn_rootkit.php
Automated approaches for detection
2015/16 CSF - Nuno Santos 49
} Each rootkit differs slightly in its approach to hiding the presence of malware and therefore requires us to take a slightly different approach to detect it
} An example of software that automates the search for known rootkits is the Chkrootkit toolkit } Runs on a dozen different UNIX platforms } Recognizes more that fifty different rootkits } Looks for deleted login / logout records, signatures of replaced system
utilities, rootkit configuration files and directories, missing processes, and signs of kernel-level subversion
Detection techniques for each rootkit type
2015/16 CSF - Nuno Santos 50
} Command-level rootkits
} Library-level rootkits
} Kernel-level rootkits
Detection of library-level rootkits
2015/16 CSF - Nuno Santos 51
} Strawman: check library hashes. But, would an MD5 or SHA-1 hash reveal library modifications?
} Not necessarily: while the runtime linker uses the low-level open() system call when it accesses the modified library file, md5sum use the fopen() library routine } Therefore, can be redirected to the unmodified library file
} Workaround: rootkit detection tools need to carry their own trusted copy of the system library routines
Detection techniques for each rootkit type
2015/16 CSF - Nuno Santos 52
} Command-level rootkits
} Library-level rootkits
} Kernel-level rootkits
Detection of kernel-level rootkits
2015/16 CSF - Nuno Santos 53
} Kernel rootkits may be exposed because they introduce little inconsistencies into a system
} Some may show up externally, in the results from system calls that manipulate processes, files, kernel modules, etc.
} Others show up only internally, in the contents of kernel data structures } E.g., hidden objects occupy some storage even though the
storage does not appear in kernel symbol tables
Inconsistencies that may reveal kernel rootkits
2015/16 CSF - Nuno Santos 54
} Output of tools that bypass the file system can reveal information that is hidden by compromised FS code } E.g., TSK
} Oversight } E.g., the modification time of an important system directory is
changed, not to the contents of that directory
} Inconsistencies in the results from process-manipulating system calls and from the /proc file system } E.g., in reporting a process as “not found”)
Inconsistencies that may reveal kernel rootkits
2015/16 CSF - Nuno Santos 55
} Unexpected behavior of some system calls } E.g., when the Adore rootkit is installed, setuid() – change process
privileges – will report success for some parameter value even though the user does not have sufficient privileges
} E.g., when the Knark rootkit is installed, settimeofday() – set the system clock – will report suffess for some parameter values even though is should always fail when invoked by an unprivileged user
} Directory hard link count inconsistencies } The hard link count of a directory, as reported by stat(), should
equal the number of subdirectories, as reported by getdents() } If a directory is being hidden, it may show up as a missing hard link
Inconsistencies that may reveal kernel rootkits
2015/16 CSF - Nuno Santos 56
} Modifications to kernel tables, such as system call table or the virtual FS table } May be detected after the fact by reading kernel memory
via /dev/kmem } Or by examining kernel memory from inside with a forensic
kernel module such as Carbonite
} Modifications to kernel tables or kernel code may be detected using a kernel module that samples critical data structures periodically
Inconsistencies that may reveal kernel rootkits
2015/16 CSF - Nuno Santos 57
} Checking against raw data sources } Modifications that hide files can show up as inconsistencies
between information from the raw disc device and information returned by the kernel file system code
} Modifications that hide network ports, processes, or kernel modules may be exposed by reading kernel memory and comparing the contents of kernel data structures with results from system calls
Example of kernel toolkit detector tool
2015/16 CSF - Nuno Santos 58
} Findrootkit: examines kernel mem through /dev/kmem and checks consistency from multiple sources: } The /dev/ksyms symbol table, with kernel function and data
addresses and sizes } The in-kernel module list, with executable code and data
segment addresses and sizes of loaded kernel modules } The in-kernel “text arena” table, with executable code
segment addresses and sizes } The function addresses in the system call jump tables and in
file system operation jump tables } The executable code and data segment sizes, as specified in
the symbol tables of kernel module files
Example of kernel toolkit detector tool
2015/16 CSF - Nuno Santos 59
} Findrootkit can produce modification reports } Report example for a Solaris kernel
} Changes to (a) the file system operations table, and (b) the system-call jump table
Conclusions
2015/16 CSF - Nuno Santos 60
} Many attacks to operating systems are performed through rootkit software
} There are three different types of rootkits depending on the OS layer targeted by the rootkit: command-, library-, and kernel-level rootkits
} Depending on the rootkit, the forensic analyst needs to employ different rootkit-detection techniques
References
2015/16 CSF - Nuno Santos
} Primary bibliography } Dan Farmer, Wietse Venema, Forensic Discovery, Chapter 5
61
Next class
CSF - Nuno Santos
} Mobile forensics: Android
2015/16 62