log2timeline - SANS · PDF file• SANS certifications: GCIA, GCIH, GCFA gold ... use Log2Timeline; # import the library that contains the log2timeline engine my $l = Log2Timeline->new(

log2timeline

- helping you to create super timelines since 2009 -

Kristinn Guðjónsson The 2011 Digital Forensics and Incident Response Summit

Austin, TX, 2011

SANS 2011 Digital Forensics and Incident Response Summit

Who am I?

• M.Sc. in computer and communication network engineering

• Worked in forensics and information security since 2005

• SANS certifications: GCIA, GCIH, GCFA gold

• SANS mentor

• Author of log2timeline

• Blog author at the SANS forensics blog

• Author of the blog: blog.kiddaland.net


• List of timestamps with associated data

▫ Extracted from multiple sources

Filesystem

Registry (Windows)

Log files, metadata, …

• Why?

▫ We are trying to tell a story.

▫ Temporal proximity.

▫ Data correlation.

Super Timeline?


Example Super Timeline Date Description

Fri Jan 16 2009 23:15:20

[SetupAPI Log] (Entry written) DriverContext: Reported hardware ID(s) from device parent bus. … [USBSTOR/DISK&VEN_M-SYS&PROD_DELL_MEMORY_KEY&REV_4.50/086086412140E1C2&0]… [USBSTOR/DISK&VEN_M-SYS&PROD_DELL_MEMORY_KEY&REV_4.50/086086412140E1C2&0]. Warning: [STORAGE/RemovableMedia/7&1ad0a3a9&0&RM]…

Fri Jan 16 2009 23:18:10

[Shortcut LNK] (Modified/Access/Created) E:/Blue Harvest Business Plan v1.doc <-./Documents and Settings/Donald Blake/Recent/Blue Harvest Business Plan v1.lnk- which is stored on a local vol type - Removable- SN 0xf434f590 - …

Fri Jan 16 2009 23:18:15

[Shortcut LNK] (Modified/Access/Created) E:/CONFIDENTIAL_SPREADSHEETS.zip <-./Documents and Settings/Donald Blake/Recent/CONFIDENTIAL_SPREADSHEETS.lnk- …

Fri Jan 16 2009 23:18:19

[Shortcut LNK] (Modified/Access/Created) E:/TIVO Research - CONFIDENTIAL.doc <-./Documents and Settings/Donald Blake/Recent/TIVO Research - CONFIDENTIAL.lnk…

Fri Jan 16 2009 23:18:19

[Shortcut LNK] (Modified/Access/Created) E:/ <-./Documents and Settings/Donald Blake/Recent/DBlake Personal (E).lnk…

Fri Jan 16 2009 23:18:26

[Internet Explorer] (index.dat creation time/Last Access) User: Donald Blake URL:file:///E:/Blue Harvest Business Plan v1.doc (file: ./Documents and Settings/Donald Blake/Local Settings/History/History.IE5/MSHist012009011220090119/index.dat)

Fri Jan 16 2009 23:18:26

[Internet Explorer] (Last Access) User: Donald Blake URL:file:///E:/Blue Harvest Business Plan v1.doc (file: ./Documents and Settings/Donald Blake/Local Settings/History/History.IE5/index.dat)

Fri Jan 16 2009 23:18:26 /Documents and Settings/Donald Blake/Recent/Blue Harvest Business Plan v1.lnk


Example Super Timeline


Brief History


Brief History


Brief History


Brief History

…and then came version 0.60

aka the killer dwarf release


• Engine rewritten

▫ Front-end separated

▫ Logic in engine

• More of an object-oriented approach

▫ Input modules inherit parent module

▫ Makes it easier to add modules

• Pre-processing libraries introduced.

• New modules and other enhancements.

Version 0.60 - today


• 43 input modules

• 11 output modules

• 2 pre-processing modules

Version 0.60

apache2_ access

apache2_ error

chrome encase_ dirlisting

evt/evtx jp_ntfs_change

exif

ff_ bookmark

firefox2 firefox3 ftk_ dirlisting

generic_ linux

iehistory iis

isatxt mactime mcafee mft mssql_ errlog

ntuser opera

oxml pcap pdf prefetch recycler restore safari

sam security setupapi skype_sql software sol squid

syslog system tln volatility win_link wmiprov xpfirewall


• Prior versions

▫ Logic in front-end

▫ Code replicated in different front-ends

▫ Input modules opened files

▫ Each file opened twice

• New structure

▫ Engine separated, logic there

▫ Front-end parses parameters

▫ Engine opens files

Changes in Structure


#!/usr/bin/perl use Log2Timeline; # import the library that contains the log2timeline engine my $l = Log2Timeline->new( „file‟ => '/mnt/analyze', # point to the file/directory to parse ‟ „recursive' => 1, # we want to recursively go through stuf #'hostname' => '', # to include a hostname (done in preprocessing) 'input' => 'winxp', # which input modules to use (this is a Win XP machine) 'output' => 'csv', # what is the output module to be used #'offset' => 0, # the time offset (if the time is wrong) 2996 #'exclusions' => '', # an exclusion list of one exists #'text' => '', # text to prepend to path of files (like c:) #'append' => 0, # we are appending to an output file, instead of writing a new one 'time_zone' => 'CST6CDT', # the time zone of the image 'preprocess' => 1, # turn on pre-processing modules ) or die( 'unable to start log2timeline'); $l->start; sub print_line($) { my $line = shift; print $line; }

How to Create a Front-end?


• Gather information prior to running

▫ Not associated with timestamps

▫ Share information with input modules

• Two simple modules added

▫ Time zone settings and hostname

▫ Default browser, both system and user

Pre-Processing


log2timeline -f winxp -z EST5EDT -m C: -r -p . > /cases/bodyfile Start processing file/dir [.] ... Starting to parse using input modules(s): [winxp] [PreProcessing] The default browser of user smith according to registry is: (FIREFOX.EXE) [PreProcessing] Unable to determine the default browser for user default user [PreProcessing] Unable to determine the default browser for user networkservice [PreProcessing] Unable to determine the default browser for user localservice [PreProcessing] Hostname is set to SIMTTO-LAPTOP [PreProcessing] The timezone according to registry is: (USMST) US Mountain Standard Time [PreProcessing] The timezone settings are NOT overwritten so the settings might have to be adjusted. [PreProcessing] The default system browser is: : IEXPLORE.EXE ("C:\Program Files\Internet Explorer\IEXPLORE.EXE" -nohome) Loading output file: csv

Pre-Processing


date time sourcetype user desc notes

5/13/11 3:39:57 Internet Explorer smith

URL:file:///C:/Documents%20and%20Settings/smith/My%20Documents/THIS_IS_THE_DOCUMENT.txt

Not the default browser (FIREFOX.EXE)

5/13/11 3:39:57 Internet Explorer smith URL::Host: My Computer

Not the default browser (FIREFOX.EXE)

10/22/09 15:25:52 Firefox 3 history smith

Bookmark URL Karadzic plans to boycott trial (http://news.bbc.co.uk/go/rss/-/2/hi/europe/8319869.stm) [8319869.stm] count 0

Default browser for user

Pre-Processing


• Old userassist changed to ntuser

• Behavior changed

▫ All keys inside a hive parsed

• Includes code from RegRipper

▫ And regtime

• Added modules to parse

▫ SYSTEM

▫ SOFTWARE

▫ SAM

▫ SECURITY

Registry Parsing


• Ported analyzeMFT into log2timeline

▫ Thanks to David Kovar for allowing me to do that

• $STDINFO and $FILENAME timestamps included

• Simple timestamp manipulation detection

▫ Prone to false positives/negatives

Filesystem Parser - $MFT


• Very simple first version of a Skype parser

▫ Only works on the SQLite database

▫ Grabs basic chat information

• Module to parse the output from jp

▫ Parses the NTFS change log

• Default output is now CSV

• Bug fixes and minor improvements

Is There More New Stuff?

date time sourcetype type user desc

2/12/10 14:39:47 Skype History Chat Sent

Kristinn Gudjonsson (<username>)

MSG written to Rob Lee (<user>): this is the chat message… (edited)

1/18/10 22:35:35 Skype History Chat Sent

Kristinn Gudjonsson (<username>) MSG written to Rob Lee (<user>): and I‟m talking some more….


• Version 0.60 now works on Windows

▫ Instructions on how to install in docs/INSTALL

▫ Thanks to Chris Pogue for creating the install documentation

… ohh and one more thing

…but how do we extract those sexy

super timelines?


• Pretty tedious task

▫ Bunch of commands need to be issued

▫ Possible to write a script to make life easier

• Things can be simplified

▫ Remember the new structure of the front-end?

▫ And the new modules that are available?

Extraction Process


timescanner –z ZONE –d MNTPOINT –w BODYFILE fls –r –m C: IMAGE >> BODYFILE regtime.pl –m HKLM-SYSTEM –r MNTPOINT/WINDOWS/System32/config/system >> BODYFILE regtime.pl –m HKLM-SAM –r MNTPOINT/WINDOWS/System32/config/SAM>> BODYFILE regtime.pl –m HKLM-SECURITY–r MNTPOINT/WINDOWS/System32/config/SECURITY >> BODYFILE regtime.pl –m HKLM-SOFTWARE–r MNTPOINT/WINDOWS/System32/config/software >> BODYFILE mactime –d –b BODYYFILE –z ZONE DATE_RANGE > CSVFILE

The old method


• ntfs-3g does not show the $MFT file ▫ Need to extract the $MFT

icat myimage.dd 0 > myimage.mft log2timeline –f mft –z EST5EDT –m C: -w /cases/bodyfile.txt log2timeline –f winxp –z EST5EDT –m C: -r –p /mnt/windows_mount –w /cases/bodyfile.txt l2t_process –b /cases/bodyfile.txt 01-15-2010..01-25-2010 > /cases/timeline.txt

The new (although manual)


• Simple frontend created: log2timeline-sift

▫ Included in the extra folder

• Can be installed easily

apt-get install log2timeline-sift-perl

• Options:

▫ -i IMAGE_FILE

▫ -c CONF (default /etc/log2timeline/sift.conf)

▫ -z ZONE

▫ -w (is a Windows 7)

▫ -p NR

The new (automated SIFT)


• To extract the super timeline using the script

▫ Creates a folder called /cases/timeline

• Partition image (not a whole disk image)

log2timeline-sift –z EST5EDT –p 0 xp_dblake.dd

• Disk image:

log2timeline-sift –z EST5EDT disk_image.dd

log2timeline-sift


• Sample run log2timeline-sift.pl -z EST5EDT -i /images/xp_dblake.dd -p 0 Image file (/images/xp_dblake.dd) has not been mounted. Do you want me to mount it for you? [y|n]: y This is a partition image, let's attempt mounting it directly. Image file mounted successfully as /mnt/windows_mount Loading output file: csv [PreProcessing] Unable to determine the default browser for user donald blake [PreProcessing] Unable to determine the default browser for user default user [PreProcessing] Unable to determine the default browser for user networkservice [PreProcessing] Unable to determine the default browser for user localservice [PreProcessing] Hostname is set to ASGARD [PreProcessing] The timezone according to registry is: (EST) Eastern Standard Time [PreProcessing] The timezone settings are NOT overwritten so the settings might have to be adjusted. [PreProcessing] The default system browser is: : IEXPLORE.EXE ("C:\Program Files\Internet Explorer\iexplore.exe" -nohome) Loading output file: csv

log2timeline-sift

and then what?


• Normal super timeline contains LOT of data

▫ Finally we have something to spend time on

• Necessary to reduce the dataset

• How?

▫ Read at the speed of light

▫ Use mactime output and the script mactime

▫ Load everything into Excel and pray

▫ Use databases or Splunk

▫ The good ol‟ grep method

grep “^05\/1[2-9]\/2011” timeline.txt

Life After Collection


• Isn‟t it possible to create a tool to assist?

▫ Well yes there is…

• l2t_process added to meet this demand

▫ Included with log2timeline

▫ Works in a similar fashion as mactime

▫ Parses the CSV and TAB format of log2timeline

Is There a Life After Collection?


• Usage l2t_process –b BODYFILE [-w white] [-k dirty] [DATE_RANGE]

• What does it do you ask?

▫ Sort entries based on time

▫ Filter based on date range

▫ Removes duplicate entries

▫ Compare entries to a keyword or whitelist file

▫ Warn if it detects “suspicious” MFT entries

▫ Create scatter plots

l2t_process


$cat keyfile this_is_the $l2t_process –b timeline.txt -k keyfile > time_key.txt Building keyword list...DONE (1 keywords loaded) Total number of events that fit into the filter (got printed) = 16 Total number of duplicate entries removed = 3 Total number of events skipped due to keyword filtering = 1281973 Total number of processed entries = 1281989 Run time of the tool: 36 sec cat time_key.txt date,time,timezone,MACB,source,sourcetype,type,user,host,short,desc,version,filename,inode,notes,format,extra 04/20/2011,08:06:32,EST5EDT,...B,FILE,NTFS $MFT,$SI [...B] time,-,-,c:/Documents and Settings/smith/My Documents/THIS_IS_THE_DOCUMENT.txt,{SUSP ENTRY - timestomp? - second prec. $SI [MACB] FN rec AFTER SI rec} c:/Documents and Settings/smith/My Documents/THIS_IS_THE_DOCUMENT.txt,2,c:/Documents and Settings/smith/My Documents/THIS_IS_THE_DOCUMENT.txt,18113,-,Log2t::input::mft,- …

l2t_process - keyword


• Done through the Windows API

▫ ZwSetInformationFile

▫ NtSetInformationFile

▫ Allows setting the whole 64 bits

▫ Many tools only use second precision

▫ Timestomp from Metasploit one of those: /* it doesnt matter what the millisecond value is because the ntfs resolution for file timestamps is only up to 1s */

systemtime->wMilliseconds = 0;

• The API only changes the $STDINFO timestamp

▫ The $FILENAME is untouched

Timestamp Manipulation


• Two methods

▫ Detect timestamps that have ms equal to zero

▫ Detect timestamps where $FN occurs later than $SI

• Problems with this approach

▫ Not all files with zero ms. time are “bad”

▫ $FN timestamps are updated when files are copied or moved

• Pretty easy to fool

▫ Use methods that set the ms. to a random value

How Do We Then Detect Those Manipulations?


• Sequential MFT entry number allocation

• Malware often hides inside Windows\System32

▫ Patches update several files

▫ Malware introduces few changes

▫ “Hide in plain sight”

• What l2t_process does to detect manipulations

▫ $MFT module includes notes if entries are suspicious

▫ The –i (include) option includes suspicious entries outside the date range

▫ Maps the relationship between MFT entry nr. and creation time

Other methods

Scatter Plots

[2139] /WINDOWS/system32/evil.exe [{SUSP ENTRY - second prec. $SI [M...] FN rec AFTER SI rec} ]


• log2timline has been evolving since 2009

▫ And keeps doing that

▫ Developed on my own time Donations and feedback run tool development

• Version 0.60 allows complete super timeline creation

▫ And runs on most platforms

▫ Easy to integrate into other scripts

▫ l2t_process assists with data reduction

Summary