Embed Size (px)
Citation preview
Mastering the Super Timeline log2timeline style
Kristinn Guðjónsson The 2010 European Community Digital Forensics and Incident Response Summit
London 2010
SANS EU Forensics and Incident Response Summit
Who am I? • M.Sc. in computer and communication network engineering • Worked in forensics and information security since 2005 • SANS certifications: GCIA, GCIH, GCFA gold • SANS mentor • Author of log2timeline • Blog author at the SANS forensics blog • Author of the blog: blog.kiddaland.net
SANS EU Forensics and Incident Response Summit
Why Timeline Analysis • Find out when events took place • Temporal proximity • Often a great place to start investigations ▫ Can quickly lead you to evidence that needs further analysis ▫ Send timeline to senior analyst during acquisition
SANS EU Forensics and Incident Response Summit
Traditional Timeline Analysis • Focused around extracted timestamp from filesystems • Different meaning depending on the filesystem • Has been done for years ▫ Needs to be extended...
SANS EU Forensics and Incident Response Summit
Traditional Timeline Analysis
Date Type Meta File Name
Wed Jul 09 2008 01:47:16 ...b 391-128-1 C:/Users/Reed Richards/NTUSER.DAT
Thu Jun 18 2009 06:17:11 mac. 391-128-1 C:/Users/Reed Richards/NTUSER.DAT
Fri Jun 19 2009 05:16:57 m.c. 46477-128-4 C:/Windows/System32/winevt/Logs/Windows/System32/winevt/Logs/System.evtx
Fri Jun 19 2009 05:16:57 m.c. 46478-128-4 C:/Windows/System32/winevt/Logs/Windows/System32/winevt/Logs/Application.evtx
What does this tell you?
SANS EU Forensics and Incident Response Summit
Traditional Timeline Analysis • Can be extremly valuable ▫ Temporal proximity
• Other problems ▫ Easy to manipulate (timestomp anyone?) ▫ Sensitive to changes ▫ Not always updated
• Are there other solutions?
SANS EU Forensics and Incident Response Summit
Solutions? • Extend the timeline? ▫ Include information from within log files ▫ Difficult to alter every source of timestamp
• Visually represent the timeline? ▫ Helps in some situations
• Make a magic tool to analyse the timeline ▫ The Forensicator Pro has this ability already
SANS EU Forensics and Incident Response Summit
Methods to Extend Timeline Analysis • Manually adding timestamps to timeline ▫ Not really efficient ▫ Need to know the location and format
• Using specially crafted tools to extract timestamps ▫ Requires knowledge of multiple tools ▫ Need to know the location of each file
SANS EU Forensics and Incident Response Summit
Problems With Manual Labour • It’s not all a walk in the park ▫ Different files use different methods to store timestamps ▫ Files are stored using different formats ▫ Timestamps are stored in varying time zones.
• Analyst must recognise all these subtle differences.
SANS EU Forensics and Incident Response Summit
Enter log2timeline • log2timeline is written to address this problem ▫ A framework designed for timeline analysis
• What does it do? ▫ Extracts timestamps from various files ▫ Outputs them in various formats
• What platform does it run on? ▫ Written to be used on a Mac OS X or Linux ▫ Needs slight changes to be ported to Windows
• Summary ▫ log2timeline is written to create a super timeline ▫ …and it does it automatically…
SANS EU Forensics and Incident Response Summit
Modules • Consists of four main parts ▫ Front-end. ▫ Input module (aka a parser). ▫ Output module ▫ Shared libraries (between modules)
• Each part independent ▫ Makes adding new functionality easy
SANS EU Forensics and Incident Response Summit
Front-ends • Currently there are three available front-ends ▫ log2timeline: CLI version of the tool ▫ glog2timeline: GUI version of the tool ▫ timescanner: Recursive scanner
SANS EU Forensics and Incident Response Summit
Currently supported input modules Module Description Module Description
apache2_access Apache 2 access log apache2_error Apache 2 error log
chrome Chrome browser history oxml Open XML metadata (.docx,pptx,...)
evt Windows Event log pcap PCAP network dump files
evtx Windows Event log (Vista+) pdf PDF metadata
exif EXIF metadata prefetch Prefetch/Superfetch
ff_bookmark Bookmarks, Firefox 2 recycler Recycle bin (XP/Vista+)
firefox2 Firefox 2 browser history restore Restore points
firefox3 Firefox 3 browser history, bookmarks setupapi SetupAPI log file
iehistory IE browser history sol Flash cookies (Local Shared Object)
iis IIS log files squid Squid access log files
isatxt ISA server firewall text export tln TLN body file
mactime Mactime bodyfile userassist Various registry key from NTUSER.DAT
mcafee McAfee anti-virus log win_link Windows shortcut file
opera Opera browser history xpfirewall Firewall log files
syslog Syslog messages
SANS EU Forensics and Incident Response Summit
Output modules Module Description
beedocs Output timeline using tab-delimited file to import into BeeDocs
cef Output timeline using the ArcSight Commen Event Format (CEF)
cftl Output timeline in a XML format that can be read by CFTL
csv Output timeline using CSV (Comma Separated Value) file
mactime Output timeline using mactime format
mactime_l Output timeline using legacy version of the mactime format (version 1.x and 2.x)
simile Output timeline in a XML format that can be read by a SIMILE widget
sqlite Output timeline into a SQLite database
tln Output timeline using H. Carvey's TLN format
tlnx Output timeline using H. Carvey's TLN format in XML
SANS EU Forensics and Incident Response Summit
Version 0.51 • Version 0.51 will be released shortly • Changes introduced ▫ New input modules
Linux log file support More information extracted from registry More Windows logs included
▫ Normal bug fixes • Changes introduced in version 0.50 ▫ Possible to select input modules in timescanner ▫ Vast speed improvements in timescanner ▫ New output modules
• Older versions too dependent on mactime ▫ mactime bodyfile is still the default output format
SANS EU Forensics and Incident Response Summit
Structure • Front-end controls the flow • Input modules do most of the work ▫ Verifies if it is capable of parsing the artifact ▫ Parses it if possible and creates the timestamp object
• Output modules arrange the output and print it ▫ Uses the timestamp object to create the output
SANS EU Forensics and Incident Response Summit
Timestamp Object • Core element of the tool • Contains all the information about each timestamp • The only object that is passed along different modules • Implemented as a Perl hash
time index value type legacy
desc short source sourcetype version [notes] extra
[filename] [host] [user] [...]
SANS EU Forensics and Incident Response Summit
Example: OpenXML • How does log2timeline parse OpenXML files? • OpenXML file (docx, pptx,...) is a ZIP file ▫ Contains XML files in a prefedined structure ▫ Example structure of a Word file:
[Content_Types].xml _rels/
.rels docProps/
app.xml core.xml thumbnail.jpeg
word/ _rels/ document.xml.rels document.xml settings.xml …
SANS EU Forensics and Incident Response Summit
First Step - Verification • ZIP files have a magic value of: 0x04034b50 • ZIP files have a file name embedded in the header • log2timeline verifies magic value and checks the file name ▫ Should be [Content_Types].xml
0000000: 504b 0304 1400 0600 0800 0000 2100 d201 PK..........!... 0000010: 98f4 8801 0000 d905 0000 1300 0802 5b43 ..............[C 0000020: 6f6e 7465 6e74 5f54 7970 6573 5d2e 786d ontent_Types].xm 0000030: 6c20 a204 0228 a000 0200 0000 0000 0000 l ...(.......... 0000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
SANS EU Forensics and Incident Response Summit
Second Step: Preparation • Extracts the file _rels/.rels from the archive ▫ Contains information about the structure ▫ Reads all docProps values
… <Relationship Id="rId4" Type="http://schemas.openxmlformats.org/officeDocument/
2006/relationships/extended-properties" Target="docProps/app.xml"/> <Relationship Id="rId3" Type="http://schemas.openxmlformats.org/package/2006/
relationships/metadata/core-properties" Target="docProps/core.xml"/> …
SANS EU Forensics and Incident Response Summit
Third Step: Parsing • Goes through each XML file that contains metadata (docProps) ▫ Timestamps are stored in a ISO-8601 format ▫ Checks if any of the value is in ISO-8601 format
Creates a timestamp object containing information extracted
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> … <dc:title>My doc</dc:title> <dc:subject>I’m doing word..</dc:subject> … <cp:lastModifiedBy></cp:lastModifiedBy> <cp:revision>1</cp:revision> <dcterms:created xsi:type="dcterms:W3CDTF”>2009-09-29T13:53:00Z</dcterms:created> <dcterms:modified xsi:type="dcterms:W3CDTF”>2009-09-29T13:59:00Z</dcterms:modified> </cp:coreProperties>
SANS EU Forensics and Incident Response Summit
Fourth step: Output • The output module prints the line
1263877260|OXML|-|Rob|(Open XML Metadata) [created] (Keynotes and Expert Briefings for Web Page) - 2010 EU - Incident Response and Digital Forensic Summit - - Application: Microsoft Office Word - Company: SANS Institute - AppVersion: 12.0000|EST5EDT|File:Keynotes and Briefings Document for Web.docx inode:1536061
time timezone type macb sourcetype source desc filename
Tue Jan 19 2010 00:01:00 EST5EDT created MACB Open XML Metadata OXML
(Keynotes and Expert Briefings for Web Page) - 2010 EU - Incident Response and Digital Forensic Summit - - Application: Microsoft Office Word - Company: SANS Institute - AppVersion: 12.0000,Rob
Keynotes and Briefings Document for Web.docx
SANS EU Forensics and Incident Response Summit
How to install? • Three possibilities ▫ Compile from sources ▫ Use repositories ▫ Use distros with the tool pre-installed
• Repository is the preferred method ▫ Tool gets updated along with system updates
• Available repositories ▫ For Debian/Ubuntu (apt-get) ▫ For Fedora (yum) ▫ For BSD systems (port install), including Mac OS X (use
macports)
SANS EU Forensics and Incident Response Summit
timescanner vs. log2timeline • log2timeline ▫ Parses and extract timestamps from a single file
• timescanner ▫ Recursive scanner
Recursively go through a mount point to extract all available timestamps
▫ Possible to either select all, or a list of modules
SANS EU Forensics and Incident Response Summit
The “Magic” Behind Timescanner • Loads all selected input modules in a hash • Goes recursively through a directory that is passed to the tool ▫ Verifies the file/directory against selected input modules ▫ If the verification succeeds the file is parsed and the next file/
directory is examined
SANS EU Forensics and Incident Response Summit
timescanner • Default behaviour to use all input modules • Possible to limit the number of modules used (-f parameter)
linux apache2_access, apache2_error, pcap, syslog,
web chrome, firefox3, firefox2, ff_bookmark, opera, iehistory, iis,
winvista chrome, evt, exif, ff_bookmark, firefox3, iehistory, iis, mcafee, opera, oxml, pdf, prefetch, recycler, restore, sol, userassist, win_link, xpfirewall,
winxp chrome, evt, exif, ff_bookmark, firefox3, iehistory, iis, mcafee, opera, oxml, pdf, prefetch, recycler, restore, setupapi, sol, userassist, win_link, xpfirewall,
• Example ▫ timescanner –z local –d /mnt/suspect –f web ▫ timescanner –z local –d /mnt/suspect –f chrome,firefox2
SANS EU Forensics and Incident Response Summit
Creating a Super Timeline • Mount the image file (read-only)
sudo mount -t ntfs-3g -o ro,loop,show_sys_files /cases/vista/vista_ntfs.dd /mnt/windows_vista_mount
• Run timescanner against it timescanner -z EST5EDT –f winvista -d /mnt/windows_vista_mount -w /cases/vista/bodyfile –log /cases/vista/timescanner.log
• Run fls to get filesystem timestamps fls –r –m C: /images/windowsforensics/vista_ntfs.dd >> /cases/vista/bodyfile
• Run any other tool capable of extracting timestamps
SANS EU Forensics and Incident Response Summit
Output Methods • Ten different output modules ▫ Three for timeline visualization ▫ Six ASCII or XML output methods ▫ One database output
SANS EU Forensics and Incident Response Summit
Visualization • Three types of visualization modules ▫ SIMILE ▫ CFTL ▫ BeeDocs
• Possible to use GnuPlot and scripts as well
SANS EU Forensics and Incident Response Summit
SIMILE Widget
• SIMILE widget is essentially a web widget for visualizing temporal data • log2timeline can export both a XML and JSON output module for SIMILE
widgets ▫ A HTML file has to be created to use and display the data though
SANS EU Forensics and Incident Response Summit
CyberForensics TimeLab (CFTL) • Created by Jens Olsson and
Martin Boldt • Not yet released, a beta version
and a research paper • Reads an image file and extracts
all timestamps using plugins • Uses XML to store and read data • log2timeline can output using a
XML file that CFTL can read
SANS EU Forensics and Incident Response Summit
CyberForensics TimeLab (CFTL)
SANS EU Forensics and Incident Response Summit
BeeDocs Visualization
• Tool to visually represent timelines in 3D among other
• Written solely for Mac OS X • Can read a tab delimited file • log2timeline can output in a
file that can be opened by BeeDocs
SANS EU Forensics and Incident Response Summit
BeeDocs
SANS EU Forensics and Incident Response Summit
GnuPlot and Scripts • Endless possibilites • Custom scripts ▫ Requiring scripting
skills • Example of a web
attack ▫ Started with a scan,
thus creating several log entries
SANS EU Forensics and Incident Response Summit
Visualization • Pros ▫ Visualization is often easier to understand ▫ Easier to explain to non-technical people ▫ Great in reports
• Cons ▫ Often extremly slow when dealing with many events ▫ Often difficult to find events of interest
• Conclusions ▫ Better to analyse using different methods ▫ Suits some investigations better than others ▫ Use visualization for reports ▫ Include limited events in the visual timeline
SANS EU Forensics and Incident Response Summit
Good Ol’ Spreadsheet • Propably the most common method • Two possibilities ▫ Use mactime output and then use mactime to convert to CSV ▫ Use the CSV output module (not sorted)
• Easy to create filters and hide fields
log2timeline -z EST5EDT -o csv -w /tmp/test.csv -f userassist NTUSER.DAT log2timeline -z EST5EDT -o csv -w /tmp/test.csv -f iehistory Local\ Settings/History/
History.IE5/index.dat
SANS EU Forensics and Incident Response Summit
Other Methods • Mandiant Highlighter • Combination of vim/less/grep using a CSV file • Or use what ever methods you can think of
SANS EU Forensics and Incident Response Summit
Review
Date Type Meta File Name
Wed Jul 09 2008 01:47:16 ...b 391-128-1 C:/Users/Reed Richards/NTUSER.DAT
Thu Jun 18 2009 06:17:11 mac. 391-128-1 C:/Users/Reed Richards/NTUSER.DAT
Fri Jun 19 2009 05:16:57 m.c. 46477-128-4 C:/Windows/System32/winevt/Logs/Windows/System32/winevt/Logs/System.evtx
Fri Jun 19 2009 05:16:57 m.c. 46478-128-4 C:/Windows/System32/winevt/Logs/Windows/System32/winevt/Logs/Application.evtx
Remember this timeline from before?
SANS EU Forensics and Incident Response Summit
Now We Have...
• Doesn’t this tell you a whole lot more?
Date Time Description
Sat Jun 13 2009 17:26:00 [System] (Event Logged) <Richards-Laptop> System/Service Control Manager ID [7036] : EventData/Data -> stopped (file: Windows/System32/winevt/Logs/System.evtx)
Sat Jun 13 2009 17:28:36 [NTUSER UserAssist key] (LastWritten) UEME_RUNPIDL [Count: 10] (file: Users/Reed Richards/NTUSER.DAT)
Sat Jun 13 2009 17:28:36 [NTUSER UserAssist key] (LastWritten) UEME_RUNPIDL:%csidl23%/iTunes/iTunes.lnk [Count: 7] (file: Users/Reed Richards/NTUSER.DAT)
Sat Jun 13 2009 17:28:37 [NTUSER UserAssist key] (LastWritten) UEME_RUNPATH:[iTunes] VIRTUAL [Count: 1] (file: /Users/Reed Richards/NTUSER.DAT)
Sat Jun 13 2009 17:33:02 [NTUSER MountPoints2 key] (Drive last mounted) Volume mounted - name: {a455ad5a-5839-11de-91f3-000000000000} (file: Users/Reed Richards/NTUSER.DAT)
Sat Jun 13 2009 17:33:12 [NTUSER UserAssist key] (LastWritten) UEME_RUNPATH:C:/Windows/system32/rundll32.exe [Count: 3] (file: Users/Reed Richards/NTUSER.DAT)
Sat Jun 13 2009 17:34:42 [System] (Event Logged) <Richards-Laptop> System/Service Control Manager ID [7036] :EventData/Data -> stopped (file: Windows/System32/winevt/Logs/System.evtx)
Sat Jun 13 2009 17:36:45
[System] (Event Logged) <Richards-Laptop> System/volsnap ID [33] :EventData/Data -> [0] /Device/HarddiskVolumeShadowCopy2[1] C:- EventData/Binary -> 00000000020030000000000021000640020000000000000001000000000000000000000000000000 (file: Windows/System32/winevt/Logs/System.evtx)
Sat Jun 13 2009 17:41:03
[Application] (Event Logged) <Richards-Laptop> Application/Application Hang ID [1002] :EventData/Data -> [0] iTunes.exe[1] 8.2.0.23[2] 888[3] 01c9ec6d5c702f60[4] 391- EventData/Binary -> 55006E006B006E006F0077006E0000000000 (file: Windows/System32/winevt/Logs/Application.evtx)
Sat Jun 13 2009 17:41:20 [Application] (Event Logged) <Richards-Laptop> Application/Desktop Window Manager ID [9009] :EventData/Data -> [0] 0x40010004- EventData/Binary -> empty (file: Windows/System32/winevt/Logs/Application.evtx)
Sat Jun 13 2009 17:41:21 [Application] (Event Logged) <Richards-Laptop> Application/Wlclntfy ID [6000] :EventData/Data -> [0] SessionEnv- EventData/Binary -> D9060000 (file: Windows/System32/winevt/Logs/Application.evtx)
Sat Jun 13 2009 17:41:21
[Application] (Event Logged) <Richards-Laptop> Application/profsvc ID [1530] :EventData/Data -> 3 user registry handles leaked from /Registry/User/S-1-5-21-865758690-3576269959-3781552731-1000:Process 836 (/Device/HarddiskVolume1/Windows/System32/svchost.exe) has opened ….Policies/Microsoft/Windows/CurrentVersion/Internet Settings (file: Windows/System32/winevt/Logs/Application.evtx)
Sat Jun 13 2009 17:41:25
[Application] (Event Logged) <Richards-Laptop> Application/profsvc ID [1530] :EventData/Data -> 1 user registry handles leaked from /Registry/User/S-1-5-21-865758690-3576269959-3781552731-1000_Classes:Process 836 (/Device/HarddiskVolume1/Windows/System32/svchost.exe) has opened key /REGISTRY/USER/S-1-5-21-865758690-3576269959-3781552731-1000_CLASSES (file: Windows/System32/winevt/Logs/Application.evtx)
SANS EU Forensics and Incident Response Summit
Future of log2timeline • New input modules ▫ Several sources that are not included
• Modify the source code so it can be used on Windows ▫ Not that much that needs to be changed
• Create a pretty GUI ▫ For those who enjoy point-and-click
• Add pre-processing ▫ Gather information to use in input modules
• Implement a test suite for validation ▫ To verify that it is working properly
• Support for image files ▫ Remove the mandatory “mount the image file first” condition
SANS EU Forensics and Incident Response Summit
Future of Super Timeline • Biggest problem is number of events ▫ Relevant entries are few needles in a large haystack
• Differences in cases require manual inspection ▫ No automatic tool yet to analyse timelines
• Need to find a way to reduce the dataset in a intuitive manner • Possibilities ▫ Use OSSEC to initially go through the timeline to find some anomalies? ▫ Use Splunk to correlate data with other sources ▫ Create a new tool that can easily filter out irrelevant entries
SANS EU Forensics and Incident Response Summit
Summary • Timelines should be extended beyond simple filesystem timestamps • Super timelines have the capability to shorten investigation time • Traditional filesystem timeline is very volatile and degrades quickly
with time ▫ The super timeline is more resilient to anti-forensics and degradation
• Super timelines can be easily created using log2timeline • log2timeline is an open-source software ▫ Developed in my own free time ▫ I like to look at it as a donation-ware
