Digital Forensics - ic.unicamp.brrocha/teaching/2013s2/mo447/classes/2013-mo44… · • Statistical analysis to detect traces of image manipulations ... Samsung L74wide, NV15 Sony

Reasoning for Complex Data (RECOD) Lab.Institute of Computing,

University of Campinas (Unicamp)

Av. Albert Einstein, 1251 - Cidade UniversitáriaCEP 13083-970 • Campinas/SP - Brasil

Digital Forensics MO447 / MC919

* Pintura de Rajib Roy, Case Investigation - 2012

Prof. Dr. Anderson Rocha

Microsoft Research Faculty FellowAffiliate Member, Brazilian Academy of Sciences

Reasoning for Complex Data (Recod) Lab.

[email protected]://www.ic.unicamp.br/~rocha

mailto:[email protected]


http://www.ic.unicamp.br/~rocha

http://www.ic.unicamp.br/~rocha

Are EXIF telltales really non-reliable?

Faculty of Computer Science Institute of Systems Architecture – Privacy and Data Security

Forensic Analysis of Ordered Data Structureson the Example of JPEG Files

Thomas Gloe ([email protected])

2012 IEEE International Workshop on Information Forensics and Security

Tenerife, 04/12/2012


Forensic Analysis of Images

The image file is the starting point for forensic investigations and

consists of image data

• Intensity and colour information• Statistical analysis to detect

traces of image manipulationsand the used source device

. e.g., sensor noise, resamplingartefacts

encapsulated in an image container format.

• Auxiliary information (imageparameters, metadata, . . . )

• Analysis allows basic check ofauthenticity and basic level ofimage source identification

. State-of-the-art methods analyseimage parameters (e.g.,dimensions, JPEG quantisationand Huffmann tables), previewimages and metadata

. Auxiliary information is oftenconsidered unreliable.

04/12/2012 Forensic Analysis of Ordered Data Structures slide 2 of 11

















































Counterfeiting Auxiliary Information of JPEGs

• The JPEG standard defines a lossy compression scheme for natural scenesand the JPEG container format.

color spaceconversion(optional)

sub-sampling(optional)

8 ⇥ 8 blocksplitting

DCT trans-formation

quantisationentropy

encodingJPEGdata

JPEGcontainer

source image

image parameters

JPEG compression

JPEG compression parameters

. Tools are available to forge auxiliary information.• Some image processing software allows to employ original JPEG

compression parameters after image manipulation (e.g., Gimp)• Alternatively, cjpeg of libJPEG allows to set all compression parameters• Metadata can be preserved (e.g., Gimp), copied (Jhead) and altered (ExifTool)• Update of preview images employing cjpeg and Jhead

. Are there other forensic characteristics in JPEG files?







DCT trans-formation

quantisationentropy

encodingJPEGdata

JPEGcontainer

source image

image parameters

JPEG compression



compression parameters after image manipulation (e.g., Gimp)• Alternatively, cjpeg of libJPEG allows to set all compression parameters

• Metadata can be preserved (e.g., Gimp), copied (Jhead) and altered (ExifTool)• Update of preview images employing cjpeg and Jhead








DCT trans-formation

quantisationentropy

encodingJPEGdata

JPEGcontainer

source image

image parameters

JPEG compression


+ metadata metadatacontainer


compression parameters after image manipulation (e.g., Gimp)• Alternatively, cjpeg of libJPEG allows to set all compression parameters• Metadata can be preserved (e.g., Gimp), copied (Jhead) and altered (ExifTool)

• Update of preview images employing cjpeg and Jhead








DCT trans-formation

quantisationentropy

encodingJPEGdata

JPEGcontainer

source image

image parameters

JPEG compression


+ preview image(s)previewJPEGdata

JPEGcontainer

preview parameters











DCT trans-formation

quantisationentropy

encodingJPEGdata

JPEGcontainer

source image

image parameters

JPEG compression


+ preview image(s)previewJPEGdata

JPEGcontainer

preview parameters






Data Structures of JPEG Container Formats

marker id short value JIF JFIF EXIF description

SOI 0xFF D8 ⇥ ⇥ ⇥ start of imageAPPn 0xFF En application dataAPP0 0xFF E0 ⇥ JFIF application dataAPP1 0xFF E1 ⇥ EXIF application dataDQT 0xFF DB ⇥ ⇥ ⇥ define quantisation tablesDHT 0xFF C4 (⇥) ⇥ ⇥ define Huffman tablesSOF 0xFF Cn ⇥ start of frameSOF 0xFF C0 ⇥ ⇥ baseline DCTSOS 0xFF DA ⇥ ⇥ ⇥ start of scanDRI 0xFF DD define restart intervalRSTn 0xFF Dn nth restartCOM 0xFF FE commentEOI 0xFF D9 ⇥ ⇥ ⇥ end of image

• Full-featured container format JPEG Interchange Format (JIF)• Subsets with lower complexity JFIF and JPEG/EXIF• Different types of information are stored in segments• Each segment is identified by a short value (marker) at the beginning.• Required markers to decompress an image successfully:

SOI, DQT, DHT, SOF, SOS as well as the image data

• Format standards predefine only position of SOI, SOS & EOI


Test Setup. Customised JPEG format parser to extract all format-specific characteristics• Employed images of the ‘Dresden Image Database’ capturing natural scenes

(16,958 JPEGs including 44,041 JPEG containers) and JPEG scenes (1,851JPEGs including 4,666 JPEG containers)

• Additionally, created post-processed images using common softwarepackages

make models

Agfa DC-504, DC-733s, DC-830i, Sensor505-X, Sensor530sCanon Ixus 55, Ixus 70Casio EX-Z150FujiFilm FinePix J50Kodak M1063Nikon CoolPix S710, D200, D70, D70sOlympus µ1050SWPanasonic DMC-FZ50Pentax Optio A40, Optio W60Praktica DCZ5.9Ricoh GX100Rollei RCP-7325XSSamsung L74wide, NV15Sony DSC-H50, DSC-T77, DSC-W170

software versions

cjpeg (libJPEG) 8cExifTool 8.59Gimp 2.6.11IrfanView 4.30Jhead 2.87Paint.NET 3.08PaintShop Pro 8.10, X3Photoshop (abbr. PS) CS2, CS3, CS4, CS5, Elements 9 (abbr. E9)

Sequence of JPEG Data StructuresDigital Cameras

camera model / type of thumbnail: camera model sequence of JPEG marker segments

EX-Z150 SOI APP1(EXIF) APP15(TEXT) DQT DHT SOF0 SOS EOIM1063 SOI APP1(EXIF) APP2(FPXR) n⇥ DQT SOF0 DHT DRI SOS RSTn EOI

DC-733s, DC-830i SOI APP1(EXIF) APP5 APP6 DQT DRI DHT SOF0 SOS EOISensor505-X, EX-Z150, CoolPix S710, Optio A40, Optio W60,DCZ5.9, L74wide, H50, T77, W170

SOI APP1(EXIF) DQT DHT SOF0 SOS EOI

DC-504, Sensor530s SOI APP1(EXIF) DQT SOF0 DHT COM SOS EOIFZ50 SOI APP1(EXIF) DQT SOF0 DHT DRI SOS EOI

DC-504, Sensor530s, Ixus 55, Ixus 70, FinePix J50, D200, D70,D70s, µ1050SW, GX100, RCP-7325XS, NV15

SOI APP1(EXIF) DQT SOF0 DHT SOS EOI

EXIF maker notes: Optio W60 SOI DHT DQT SOF0 SOS EOIEXIF IFD1: DC-733s, DC-830i, Sensor505-X, EX-Z150, CoolPixS710, Optio A40, Optio W60, DCZ5.9, L74wide, H50, T77,W170; EXIF maker notes: EX-Z150, CoolPix S710, Optio A40;post thumb: L74wide

SOI DQT DHT SOF0 SOS EOI

EXIF IFD1: DC-504, Sensor530s SOI DQT SOF0 DHT COM SOS EOIEXIF IFD1: Ixus 55, Ixus 70, FinePix J50, M1063, D200, D70,D70s, µ1050SW, FZ50, GX100, RCP-7325XS, NV15; EXIF maker

notes: D200, D70, D70s, GX100, RCP-7325XS; APP2(FPXR):

M1063; post thumb: µ1050SW, NV15

SOI DQT SOF0 DHT SOS EOI

• Sequence and occurrence of marker segments differs between groups ofmodels

• Most cameras store one quantisation table for intensity (Y) and one for colourinformation (Cb/Cr) (exceptions are for example Ricoh GX100, Pentax W60)


Sequence of JPEG Data StructuresImage Processing Software

software / type of thumbnail:software

sequence of JPEG marker segments (selection)

PS CS2, CS3, CS4, E9 SOI APP0(JFIF) APP1(EXIF) APP13(PS3) APP1(XMP) APP2(ICC) APP14(Adobe) DQT SOF0 DRI DHT SOS RSTn EOIPS CS2, CS3, CS4, E9 SOI APP0(JFIF) APP1(EXIF) APP13(PS3) APP1(XMP) APP2(ICC) APP14(Adobe) DQT SOF2 DHT SOS SOS SOS ...

Gimp (with original compres-sion settings), Paint.Net

SOI APP0(JFIF) APP1(EXIF) APP2(ICC) DQT DQT SOF0 DHT DHT DHT DHT SOS EOI

Gimp SOI APP0(JFIF) APP1(EXIF) COM DQT DQT SOF0 DHT DHT DHT DHT DRI SOS EOI

Gimp, IrfanView, Paint.Net SOI APP0(JFIF) APP1(EXIF) DQT DQT SOF0 DHT DHT DHT DHT SOS EOIGimp, IrfanView SOI APP0(JFIF) APP1(EXIF) DQT DQT SOF2 DHT DHT SOS DHT SOS DHT SOS ...

IrfanView SOI APP0(JFIF) APP1(EXIF) DQT SOF0 DHT DHT SOS EOIIrfanView SOI APP0(JFIF) APP1(EXIF) DQT SOF2 DHT SOS DHT SOS DHT SOS DHT SOS ...

Gimp SOI APP0(JFIF) COM DQT DQT SOF0 DHT DHT DHT DHT SOS EOIPaintShop Pro 8.10 SOI APP0(JFIF) COM SOF0 DQT DHT SOS EOI

PaintShop Pro 8.10, X3 SOI APP0(JFIF) COM SOF2 DQT DHT DHT DHT SOS DHT SOS DHT SOS ...cjpeg (libJPEG), Gimp, Irfan-View / EXIF IFD1: Gimp

SOI APP0(JFIF) DQT DQT SOF0 DHT DHT DHT DHT SOS EOI

PaintShop Pro X3 SOI APP0(JFIF) SOF3 DHT DHT DHT SOS EOIPS CS5 SOI APP1(EXIF) APP13(PS3) APP1(XMP) APP2(ICC) APP14(Adobe) DQT SOF0 DRI DHT SOS RSTn EOI

PS CS5 SOI APP1(EXIF) APP13(PS3) APP1(XMP) APP2(ICC) APP14(Adobe) DQT SOF2 DHT SOS SOS SOS SOS ...PaintShop Pro 8.10, X3 SOI APP1(EXIF) SOF0 DQT DHT SOS EOI

APP13(PS3)& EXIF IFD1: PSCS2, CS3, CS4, E9

SOI APP0(JFIF) APP13(ACM) APP14(Adobe) DQT SOF0 DRI DHT SOS RSTn EOI

APP13(PS3)& EXIF IFD1: PSCS5

SOI APP13(ACM) APP14(Adobe) DQT SOF0 DRI DHT SOS RSTn EOI

EXIF IFD1: PaintShop Pro X3 SOI SOF0 DQT DHT SOS EOI

• Image processing software employs different sequences marker segments

• Gimp and cjpeg allow to employ original compression settings, but usedifferent sequences of marker segments


Sequence of JPEG Data StructuresImage Processing Software

software / type of thumbnail:software

sequence of JPEG marker segments (selection)

PS CS2, CS3, CS4, E9 SOI APP0(JFIF) APP1(EXIF) APP13(PS3) APP1(XMP) APP2(ICC) APP14(Adobe) DQT SOF0 DRI DHT SOS RSTn EOIPS CS2, CS3, CS4, E9 SOI APP0(JFIF) APP1(EXIF) APP13(PS3) APP1(XMP) APP2(ICC) APP14(Adobe) DQT SOF2 DHT SOS SOS SOS ...

Gimp (with original compres-sion settings), Paint.Net

SOI APP0(JFIF) APP1(EXIF) APP2(ICC) DQT DQT SOF0 DHT DHT DHT DHT SOS EOI

Gimp SOI APP0(JFIF) APP1(EXIF) COM DQT DQT SOF0 DHT DHT DHT DHT DRI SOS EOI

Gimp, IrfanView, Paint.Net SOI APP0(JFIF) APP1(EXIF) DQT DQT SOF0 DHT DHT DHT DHT SOS EOIGimp, IrfanView SOI APP0(JFIF) APP1(EXIF) DQT DQT SOF2 DHT DHT SOS DHT SOS DHT SOS ...

IrfanView SOI APP0(JFIF) APP1(EXIF) DQT SOF0 DHT DHT SOS EOIIrfanView SOI APP0(JFIF) APP1(EXIF) DQT SOF2 DHT SOS DHT SOS DHT SOS DHT SOS ...

Gimp SOI APP0(JFIF) COM DQT DQT SOF0 DHT DHT DHT DHT SOS EOIPaintShop Pro 8.10 SOI APP0(JFIF) COM SOF0 DQT DHT SOS EOI

PaintShop Pro 8.10, X3 SOI APP0(JFIF) COM SOF2 DQT DHT DHT DHT SOS DHT SOS DHT SOS ...cjpeg (libJPEG), Gimp, Irfan-View / EXIF IFD1: Gimp

SOI APP0(JFIF) DQT DQT SOF0 DHT DHT DHT DHT SOS EOI

PaintShop Pro X3 SOI APP0(JFIF) SOF3 DHT DHT DHT SOS EOIPS CS5 SOI APP1(EXIF) APP13(PS3) APP1(XMP) APP2(ICC) APP14(Adobe) DQT SOF0 DRI DHT SOS RSTn EOI

PS CS5 SOI APP1(EXIF) APP13(PS3) APP1(XMP) APP2(ICC) APP14(Adobe) DQT SOF2 DHT SOS SOS SOS SOS ...PaintShop Pro 8.10, X3 SOI APP1(EXIF) SOF0 DQT DHT SOS EOI

APP13(PS3)& EXIF IFD1: PSCS2, CS3, CS4, E9

SOI APP0(JFIF) APP13(ACM) APP14(Adobe) DQT SOF0 DRI DHT SOS RSTn EOI

APP13(PS3)& EXIF IFD1: PSCS5

SOI APP13(ACM) APP14(Adobe) DQT SOF0 DRI DHT SOS RSTn EOI

EXIF IFD1: PaintShop Pro X3 SOI SOF0 DQT DHT SOS EOI

• Image processing software employs different sequences marker segments• Gimp and cjpeg allow to employ original compression settings, but use

different sequences of marker segments


Data Structures of EXIF Metadata• Metadata stores acquisition parameters, time, coordinates, preview images,

. . .• EXIF metadata format is based on TIFF

entry group content

EXIF identifier character string ‘Exif’TIFF header byte order (little- or big-endian), number 42, offset to 0th IFD

0th IFD entries of 0th IFD defining general image properties, like image dimension,and offsets to other IFDs: EXIF IFD, GPS IFD (optional), manufacturer non-standardised IFDs (optional) and 1st IFD storing the standard thumbnail

0th IFD data storage for data of 0th IFD greater than 32 bit

EXIF IFD entries of EXIF IFD including version, camera settings and manufacturer-specific maker notes

EXIF IFD data storage for data of EXIF IFD greater than 32 bit

GPS IFD entries specific to GPS IFD including GPS coordinatesGPS IFD data storage for data of GPS IFD greater than 32 bit

man. IFD entries specific to manufacturer non-standardised IFDs

man. IFD data storage for data of man. IFD greater than 32 bit

1st IFD entries specific to 1st IFD describing the thumbnail properties1st IFD data storage for data of 1st IFD greater than 32 bit

thumbnail thumbnail data with its own sequence of JPEG marker segments


Data Structures of EXIF Metadata• Structure of standardised EXIF entries in 0th, Exif, GPS and 1st IFD:

tag data type count value or offsetto IFD data

byte1 2 3 4 5 6 7 8 9 10 11 12

• Tag identifies the semantical meaning of information stored within an entry(similar to JPEG markers).

• Standard suggests for each tag one or more types• Count specifies the number of values• Values are either stored directly, when all values together are 32 bit, or at

an offset position within the corresponding data segment.

. Exif standard proposes to store the sequence of entries in accordance to theirtag number.

. Sequence of data stored in the data segment is not standardised and differsbetween camera and software manufacturers.


Editing EXIF Metadata• Selection of EXIF entries of Canon Ixus 70 before and after metadata editing:

tag tag interpreted data type count offset value

entries of 0th IFD – authentic camera image0x010F Make 2 6 122 Canon0x0110 Model 2 22 128 Canon DIGITAL IXUS 700x0112 Orientation 3 1 - 1 (top + left-hand)0x011A XResolution 5 1 160 180/10x011B YResolution 5 1 168 180/10x0128 ResolutionUnit 3 1 - 2 (inches)0x0132 DateTime 2 20 176 2009:01:07 20:14:510x0213 YCbCrPositioning 3 1 - 1 (centred)0x8769 EXIF IFD pointer 4 1 186 -

entries of 0th IFD – tag DateTime edited with exiftool (all differences are bold)0x010F Make 2 6 122 Canon0x0110 Model 2 22 128 Canon DIGITAL IXUS 700x0112 Orientation 3 1 - 1 (top + left-hand)0x011A XResolution 5 1 150 180/10x011B YResolution 5 1 158 180/10x0128 ResolutionUnit 3 1 - 2 (inches)0x0132 DateTime 2 20 166 2012:07:01 23:59:59

0x0213 YCbCrPositioning 3 1 - 1 (centred)0x8769 EXIF IFD pointer 4 1 186 -

• Sequence of EXIF entries (tags), data type and offset before the first previewimage is equal for all images acquired with the same camera model

. Updating entries with ExifTool reorders values in the data segment

. Detecting manipulations of EXIF metadata is possible




















Summary

• File format standards are boring to read, but we can use their complexity todistinguish between files acquired with different devices and files stored withprocessing software.

• Order and occurrence of data structures differs between implementations(e.g., JPEG files and EXIF metadata) and makes auxiliary information morereliably than commonly accepted

• Simple characteristics of the JPEG container format for quick separation ofauthentic and manipulated images

– Investigated digital cameras store natural images in JPEG/EXIF format.Characteristic start of file: [SOI, APP1, segment length, identifier ‘Exif’]

– Image processing software stores JPEG files typically in JFIF.Characteristic start of file: [SOI, APP0, segment length, identifier ‘JFIF’]

. Some image processing software adds software-specific segments.e.g., Photoshop APP13 & APP14 (thumbnail image, metadata in the XMP-Format, ICC-Profil, . . . )

. Perfect forgeries are possible, but at the moment no software or combinationof software is available preserving all characteristics


Faculty of Computer Science Institute of Systems Architecture – Privacy and Data Security

Forensic Analysis of Ordered Data Structureson the Example of JPEG Files

Questions or Comments?

‘Dresden Image Database’ . https:\\forensics.inf.tu-dresden.de/ddimgdb

Thomas Gloe ([email protected])

Tenerife, 04/12/2012


Documents

Digital Forensics - ic.unicamp.brrocha/teaching/2013s2/mo447/classes/2013-mo44… · • Statistical analysis to detect traces of image manipulations ... Samsung L74wide, NV15 Sony