Embed Size (px)
Citation preview
Web and Social Media Image Forensics for News Professionals
Markos Zampoglou, Symeon Papadopoulos, Yiannis Kompatsiaris
Centre for Research and Technology Hellas (CERTH) – Information Technologies Institute (ITI),
Thessaloniki Greece
Ruben Bouwmeester, Jochen Spangenberg
Deutsche Welle, Bonn/Berlin, Germany
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Social media for news reporting (1/2)• “Eyewitness media”: an essential component in news
reporting– Photos/videos/audio of unfolding events captured by non-
professionals – Mostly disseminated via social media
• News agencies cannot shun such content, or they might:– miss out on “being first” – lose opportunity to establish contact with sources– lose important sides of the story– miss out on opportunities for user feedback
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Social media for news reporting (2/2)• But, eyewitness media can be untrustworthy
– Material may be manipulated to serve personal ambition or propaganda
– e.g. reusing past images from unrelated contexts, or explicitly tampering with media content
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Sharks in the streets of Puerto Rico (or New Jersey, or Houston…)
Picture of a Moroccan woman claimed to show a Paris attacker
Image verification: tools of the trade
• Metadata analysis– E.g. do the dates/locations match? Is the image already
copyrighted? By whom?• Reverse image search using e.g. Google or TinEye
– Has the image been posted elsewhere? Does it originate from a different context?
• Content analysis for tampering localization– Most commonly, Error Level Analysis (ELA)
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Tampering localization algorithms• Copy-moving / cloning
– Search for self-similarities within the image– Minimize the computational cost of search– Maximize flexibility with respect to transformations
• Splicing– Search for trace inconsistencies within the image– Color Filter Array patterns, JPEG DCT coefficients, camera noise patterns– Can be used to detect many copy-move forgeries as well
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
The pitfalls of image forensics
“MH17 – Forensic Analysis of Satellite Images Released by the Russian Ministry of Defence”
• A report by Bellingcat1 using (amongst other tools) ELA to prove that the Russian MoD had forged the images
• N. Krawetz: Bellingcat “misinterpreted the results”2
1https://www.bellingcat.com 2http://www.hackerfactor.com/blog/index.php?/archives/676-Continuing-Education.html
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Image verification software: requirements
• Easy to use; requires little expertise and training• Integrates easily in established workflows and IT systems• Provides all common functionalities
– Metadata analysis– Reverse image search– Tampering localization
• Clear, interpretable and transparent output– No “black box” results
• Intuitive and simple interface• Allows sharing and archiving of reports and conclusions• Comparable or better results than existing solutions
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
REVEAL Media Verification Assistant (1/2)
http://reveal-mklab.iti.gr/• Currently in the alpha stage of development• Continuous collaboration between professionals from computer
engineering (CERTH) and journalism (DW)• Features:
– Metadata: listing, GPS geolocation, EXIF thumbnail extraction– Reverse image search: Google Images integration– Tampering detection: implementation of six state-of-the-art algorithms
(ELA; Median Noise; Farid, 2009; Ferrara et al, 2012; Lin et al, 2009; Mahdian et al, 2009)
• Aims:– Provide a comprehensive, self-contained toolset for image verification– Serve as an evaluation framework for verification tools and protocols
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
REVEAL Media Verification Assistant (2/2)
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Comparison of verification tools
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
FotoForensics1 Forensically2 Ghiro3 REVEAL4
ELA X X X X
Ghost X
DW Noise X
Median Noise X X
Block Artifact X
Double Quantization X
Copy-move X
Thumbnail X X
Metadata X X X X
Geotagging X X X X
Reverse search X
1http://fotoforensics.com 2http://29a.ch/photo-forensics/ 3http://www.imageforensic.org/ 4http://reveal-mklab.iti.gr/reveal/verify.html
Further features
• Free to use• Open source1
• Real-world examples of detections and non-detections• Tutorials and explanations of algorithm mechanics• “Magnifying glass” for examining fine details• Export analysis as a personalized PDF document
including all data and comments by the investigator
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
1https://github.com/MKLab-ITI/image-forensics
Current challenges (1/2)• Are new protocols necessary?
– E.g. automatic cross-checking of specific metadata fields• Scalability is still an open issue• Interpretation of results is not always intuitive, even in
untampered images
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Current challenges (2/2)
• When applied in real-world cases, algorithms perform significantly worse than in research datasets– This could partly be attributed to the effects of Social Media
dissemination
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Quantitative evaluations (1/3)
• To assess the performance of the implemented algorithms, we evaluated their performance on a large number of images
• Benchmark datasets:– Columbia Uncompressed (Hsu and Chang 2006, “COLUMB”), First
Image Forensics Challenge (“CHAL”)1, Fontani et al 2013 (“FON”)• Additional datasets
– The Wild Web Dataset (Zampoglou et al, 2015)– The Deutsche Welle Image Forensics Dataset
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
1http://ifc.recod.ic.unicamp.br/fc
Quantitative evaluations (2/3)
• Evaluations on benchmark datasets:
• Robustness to resaves:
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Quantitative evaluations (3/3)
• Performance on the Wild Web Dataset:
• Performance on the Deutsche Welle Image Forensics Dataset:– Successful detection for 2 out of 6 cases (4 out of 12 images)
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Algorithm DQ GHO BLK ELA DWHF MEDDetections (out of 80) 3 12 3 2 3 1
Time (sec) 0.27 6.12 13.40 1.29 122.13 0.54
Conclusions• Automatic evaluations demonstrate certain limitations
in the state-of-the-art in tampering localization• But:
– Automatic evaluations not the same as human investigation– Tampering localization algorithms are only one verification
tool alongside reverse search and metadata analysis• The REVEAL Media Verification Assistant combines
virtually all known methods currently used for Social Media verification
• So, what next?
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Future steps
• Re-examine the requirements using more formal processes involving large numbers of people
• Evaluate the implemented methods using user feedback instead of automated processes
• Extend the current toolset with novel functionalities– Make outputs more easy to interpret– Keep up-to-date with the state-of-the-art– Incorporate novel proposals by investigators
• Keep improving stability, speed, GUI responsiveness• It will all depend on feedback!
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
References• Ferrara, Pasquale, Tiziano Bianchi, Alessia De Rosa, and Alessandro Piva. "Image forgery
localization via fine-grained analysis of cfa artifacts." Information Forensics and Security, IEEE Transactions on 7, no. 5 (2012): 1566-1577.
• Farid, Hany. "Exposing digital forgeries from JPEG ghosts." Information Forensics and Security, IEEE Transactions on 4, no. 1 (2009): 154-160.
• Fontani, Marco, Tiziano Bianchi, Alessia De Rosa, Alessandro Piva, and Mauro Barni. "A framework for decision fusion in image forensics based on dempster–shafer theory of evidence." Information Forensics and Security, IEEE Transactions on 8, no. 4 (2013): 593-607.
• Lin, Zhouchen, Junfeng He, Xiaoou Tang, and Chi-Keung Tang. "Fast, automatic and fine-grained tampered JPEG image detection via DCT coefficient analysis." Pattern Recognition 42, no. 11 (2009): 2492-2501.
• Mahdian, Babak and Stanislav Saic, “Using noise inconsistencies for blind image forensics,” Image and Vision Computing, vol. 27, no. 10, pp. 1497–1503, 2009.
• Zampoglou, Markos, Symeon Papadopoulos, and Yiannis Kompatsiaris. "Detecting image splicing in the wild (WEB)." In Multimedia & Expo Workshops (ICMEW), 2015 IEEE International Conference on, pp. 1-6. IEEE, 2015.
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
Thank you!
Get in touch:[email protected]@[email protected]@dw.com
http://reveal-mklab.iti.gr/
Social Media in the Newsroom Workshop, #SMnews @ ICWSM 2016 | 17 May 2016 | Cologne, Germany
![Adversarial Learning for Image Forensics Deep Matching with ...arXiv:1809.02791v1 [cs.CV] 8 Sep 2018 1 Adversarial Learning for Image Forensics Deep Matching with Atrous Convolution](https://img.pdfslide.us/doc/110x75/5ff3826adb395759682b95e7/adversarial-learning-for-image-forensics-deep-matching-with-arxiv180902791v1.jpg)
