Dark Dataand

Missing Evidence

Rob ZirnsteinPresident

Forensic InnovationsJanuary 13th, 2011

Darth Vader?

• No, “Dark Data”, but they both– Are often associated with evil– Keep secrets (“Luke, I’m your father”)– Are potentially harmful

Dark Matter?

• No, “Dark Data”! But they both– Go undetected– Are surrounded by detectable stuff– Affect things around them

What is Dark Data?

• Dark Data in our digital devices– Everyone creates it (unintentionally)– Criminals may hide it (Anti-Forensics)– Forensic tools can’t see it– But it is there!

• Data that we can’t see– On our hard drives– On out flash drives– In our computer files

Where is Dark Data?

• DCO & HPA• Unformatted Disk Space• Deleted Files• Unknown Files• Between Files• Inside Common Files• Deleted Data Objects

Hard Drive Layout

• Device Configuration Overlay (DCO)

– http://www.forensicswiki.org/wiki/SAFE_Block_XP– Data Cleaner+ http://www.mp3cdsoftware.com/blancco---data-cleaner--download-16317.htm– http://www.utica.edu/academic/institutes/ecii/publications/articles/EFE36584-D13F-2962-67BE

B146864A2671.pdf

• Host Protected Area (HPA)

– http://www.thinkwiki.org/wiki/Hidden_Protected_Area– Forensic Duplicator http://www.tableau.com/pdf/en/Tableau_TD1_Product_Brief.pdf– HDD Capacity Restore Tool http://hddguru.com/software/2007.07.20-HDD-Capacity-

Restore-Tool/

• Unformatted Disk Space

Deleted Files

• Deleted Files aren’t really gone?– Unused Disk Space (in a volume)– Disk Caches / Swap Files– Windows Recycle Bin

• Are they hard to recover?– Fragmentation is deadly– Large databases tend to be heavily fragmented– Even DFRWS Researchers find that fragmentation can make some file types impossible to recover (http://www.dfrws.org/2007/challenge/results.shtml)

Unknown Files (1)

• 500 types of files handled by eDiscovery, Document Management & Computer Forensics Tools

• 50,000+* types of files in the world• 5,000 types of files typically in use

*http://filext.com

Unknown Files (2)

Typical Tools FI Tools (23 wrong files) (26 Correct Files)

Between Files

• Alternate Data Streams (ADS)– Files hiding behind files (on NTFS)

• RAM Slack– Padding between the end of a file and the end of the

current sector– Typically zeros, sometimes random content

• File/Cluster/Residual/Drive Slack– Padding between sectors used & the end of the current cluster– Previous sector content that should be used in File Carving– http://www.forensics-intl.com/def6.html

Inside Common Files

• Deleted Objects– Ex: Adobe PDF & MS Office 2003 (OLE)

not removing deleted data (change tracking)

• Smuggled Objects– Ex: MS Office 2007 (Zip) and MS Wave

(RIFF) formats ignore foreign objects

• Object / Stream Slack– Ex: OLE objects have sector size issues,

just like with disk sectors

• Field Slack– Ex: Image files that don’t use the whole

palette, and/or less than 8/16/32/48 bpp– Steganography

Smuggled Objects

• Some formats ignoreforeign objects–MS Office 2007 (Zip)–MS Wave (RIFF)

• This example– I added a file to a

Word 2007 document.– The document opens

without any error.

Deleted Data in Slack

Deleted Data that evades Redaction

Steganography

Intentional Data Hiding

Is Dark Data Important?

• Cases are won or lost based on the ability to find the evidence.– The strongest evidence may be hidden accidentally or

intentionally.

• Corporate Digital Assets may be lost, but recoverable.

• Employee misconduct is tracked by the hidden trail of improper acts.

• Intellectual Property theft canput a company out of business.– Identify in-house criminals by detect-

ing smuggled data before it leaves.

Dark Data Can Be Fragile

– Live Forensics software tools run on the live system.• The RAM that they use affects the memory cache files

on the hard drive.• The running computer deletes, fragments & over

writes files on the hard drive constantly.• Hard drive activity can destroy Dark Data!

– Dark Data must be collected first!• Before other tools interfere with the data.

1. Image RAM2. Image Hard Drive (when possible)3. Analyze Unallocated Disk Space4. Analyze File Slack Space5. Collect relevant file types

What Does FI Do?

• Create Technologies to Capture Dark Data– File Investigator– File Expander– File Harvester

• Equip Law Enforcement with Tools– FI TOOLS– FI Object Explorer– FI Data Profiler Portable

FI Technologies

• File Investigator– Discovers Files Masquerading as Other Types– Identifies 3,953+ File Types– High Accuracy & Speed

• File Expander– Discovers Hidden Data within files– Data missed by all forensic tools

• File Harvester (Under Development)

– Recovers deleted/lost files therest of the industry can’t

– Will eventually rebuild partial files

Thank you

• ContactRob ZirnsteinRob.Zirnstein@ForensicInnovations.comwww.ForensicInnovations.com(317) 430-6891