peHash : A novel approach to Fast Malware Clustering

peHash: A novel approach to Fast Malware Clustering

By: Georg WicherskiPresenting: Rasika Bindoo

IntroductionData collection not a problem anymore

because of honeypots.Honeypots suffer from a drawback of

polluting malware databases.Anti-Viruses are slow.Thus development of peHash for clustering

group instances of the same polymorphic instances.

Other Attempts at HashingSpamsum, mrshash n-grams Signatures Vx-Class

peHash Function DesignThe function should have the following design

characteristicsIt should not have the need to look into the

contents of the sections.Low computational complexity.Scaling the result of the bzip2 compression

ratio to [0…7] С N leads to best matches.

Structural propertiesThe polymorphic malware share the same

structural Portable Executable properties.Thus following properties are taken into

account for distinction between binaries : Image characteristics. Subsystem. Stack commit size. Heap commit size.

Structural propertiesStructural information used for each section

in the Portable Executable. Virtual address Raw size Section Characteristics

Generation of hash valueshash[0] := characteristics[0…7]

V characteristics[8…15]hash[1] := subsystem[0…7]

V subsystem [8…15]hash[2] := stackcommit[0…7]

V stackcommit[8…15] V stackcommit[24…31]

hash[3] := heapcommit[0…7] V heapcommit [8…15] V heapcommit[24…31]

‘V’ symbolizes XOR operation

Generation of hash valuesSub-hash

shash[0] := virtaddress-9…31]shash[2] := rawsize[8…31]shash[4] := characteristics[16…23]

V characteristics[24…31]shash[5] := kolmogorov ϵ [0…7] С N

Advantages of this hash functionComplexity is O(1).SHA1 of the hash buffer is calculated to

obtain the final hash value. Thus difficult to create collisions.Constant length hashes are generated in

spite of variable number of sections in the executables.

Entry Points and ImportsThe value of entry point can be easily

changed for each instance of polymorphic specimen.

Most packers specify misleading Import Address Tables.

The import information can also be easily changed without any noteworthy efforts and hence not included in the hash function.

Thus both entry point information and imports are not included in hash function.

EvaluationCluster Size

MwcollectAlliance

Arbor Networks

1 7109 16543

2-9 3165 4104

10-99 549 611

100-499 70 71

500-999 19 4

1000-4999 18 8

5000+ 7 2

• peHash helps in clustering of polymorphic malware and also helps in detecting broken copies of already known threats.

EvaluationFile MD5 Size

diantz.exe48734e9b45dca36e8a…

makecab.exe2740dc2fbefaddb891f…

find.exe09b4e22c86f7e9f1e5…

print.exe76b96ed5304319f208…

subst.exe77847ef3cec784b137…

bootvrfy.exec2ab77d9dc66447dc1…

comrereg.exe908f0eda6a49625f98…

dcomcnfg.exe1178cd20b90936837d…

• Files in broken cluster share same size.

• Differentiation can be done only by looking at actual code or imports. Hence not possible for peHash.

Performance • Analysis to be carried out for one sample per peHash cluster.

• Performance is not related to binary size or section count.

ConclusionpeHash provides a performant solution to the

problem of seemingly new malware samples.peHash can accomplish correct clustering for

large sets by using basic information from Portable Executables.

peHash cannot be used to cluster variants of malware families for which code structure has to be analyzed.

Thank You

peHash : A novel approach to Fast Malware Clustering

Documents

A Novel Fast Fuzzy C-Means Clustering Technique for

A novel method of mobility-based clustering protocol in

A Novel Phase‐Unwrapping Method Based on Pixel Clustering ... articles/Cheng.pdf · FULL PAPER A Novel Phase-Unwrapping Method Based on Pixel Clustering and Local Surface Fitting

Malytics: A Malware Detection Schemeanalysis. Besides good precision and recognition rate, a malware detection scheme needs to be able to gen-eralize well for novel malware families

create yara rules with a malware zoo for hunting Python ... · Clustering vs Classification Clustering: Automatic grouping of similar objects into sets. Classification: Identifying

A Novel Framework to Classify Malware in MIPS Architecture … · 2019. 12. 31. · Research Article A Novel Framework to Classify Malware in MIPS Architecture-Based IoT Devices Tran

A Novel Image Segmentation Based on Clustering and Population … · 2020. 7. 12. · A Novel Image Segmentation Based on Clustering and Population-Based Optimisation Seyed Jalaleddin

Novel linkage disequilibrium clustering algorithm ... linkage disequilibrium clustering algorithm identifies new ... Abstract Systemic lupus erythematosus ... Novel linkage disequilibrium

Machine Learning for Malware Classification and Clustering

Scalable Fine-Grained Behavioral Clustering of HTTP-Based … fine... · 2020. 7. 24. · Scalable Fine-Grained Behavioral Clustering of HTTP-Based Malware Roberto Perdiscia, Davide

On Challenges in Evaluating Malware Clustering

A Novel Clustering Technique Using Backscattering … › wp-content › uploads › sites › ...A Novel Clustering Technique Using Backscattering Side Channel for Counterfeit IC

DETECT: A Hierarchical Clustering Algorithm for ... · ‘DETECT’ (Detection of Educational Trends Elicited by Clustering Time-series data), a novel divisive hierarchical clustering

A Novel Energy Efficient, Distributed, Clustering Based Network

Clustering for hybrid malware analysis and multi-path execution...Manual malware analysis is not ﬃt due to the huge number of such cases. As a result automated yet ﬃt malware analysis

Geosocial Co-Clustering: A Novel Framework for Geosocial …dm.kaist.ac.kr/jaegil/papers/geocode_full.pdf · 2017. 9. 27. · Geosocial Co-Clustering: A Novel Framework for Geosocial

A Novel Fuzzy Clustering Approach Based on Breadth-first

Malware clustering and classification Peng Li pengli@cs.unc.edu

A Novel Graph based Fuzzy Clustering Technique for ......A Novel Graph based Fuzzy Clustering Technique for Unsupervised Classi cation of Remote Sensing Images Biplab Banerjee and

Behavioral Clustering of HTTP-based Malware and … · and Signature Generation using Malicious Network Traces ... Honeypot GET /in.php?affid ... Detecting Web-based Malware FW Web-Proxy