• Home

  • Documents

  • AI Based An$virus: Detec$ng Android Malware Variants With a
13
AI Based An*virus: Detec*ng Android Malware Variants With a Deep Learning System Thomas Lei Wang @thomaslwang

AI Based An$virus: Detec$ng Android Malware Variants With a

Embed Size (px)

Citation preview

Page 1: AI Based An$virus: Detec$ng Android Malware Variants With a

AIBasedAn*virus:Detec*ngAndroidMalwareVariantsWithaDeepLearningSystem

ThomasLeiWang@thomaslwang

Page 2: AI Based An$virus: Detec$ng Android Malware Variants With a

About me • My first (boring) job was a virus analyst in 2004. •  I had a dream…

Page 3: AI Based An$virus: Detec$ng Android Malware Variants With a

Virus Analysis VS Image Recognition

Experiencedvirusanalystsome*mesisdoingimagerecogni*on!

ImageProvidedbytheMNISThandwriHendatabase

Page 4: AI Based An$virus: Detec$ng Android Malware Variants With a

Sample increase VS signature efficiency decrease

NumberofMaliciousAndroidApps Dowgin:ARichVariantsAndroidAdwareFamilyNewDowginSamplesVSAverageDowginSamplesHitPerSignature

Maliciousapps,DowginsamplesandDowginsignaturesarecountedfromourdatabase.

Page 5: AI Based An$virus: Detec$ng Android Malware Variants With a

Signaturebasedrules

Behavioralbasedrules

Opcodebasedrules

AIbaseddeeplearningsystem

Our evolution

Page 6: AI Based An$virus: Detec$ng Android Malware Variants With a

Training FeatureExtrac+on

•  Structuraltype

•  Sta*s*caltype •  Empiricaltype •  Continuousvalue

•  0-1value

FeatureNormaliza+on

•  Standardscorenormaliza*on

•  CuVngtechnique

•  Quan*lenormaliza*on

TraininginDeepNeuralNetwork

•  PaddlePaddleplaYorm

•  Residuallayer •  AutoEncoder •  Configura*ontunings

Models

• Malwaremodel

•  PUAmodel

InputAPKfeatures Model Output

Prediction

Page 7: AI Based An$virus: Detec$ng Android Malware Variants With a

APK

Feature extraction

Structuralfeatures• Numofuses-permissonsinAndroidManifest• Numberofpicturefilesin/res• Sizeof/res• NumberofclassesstartswithLcom/• NumofclassesstartswithLjava/• Numoffieldstypeboolean• Numofmethodswhichhasparameters>20

Empiricalfeatures• Hasexecutablefilein/res• Hasapkfilein/assets• RegisterDEVICE_ADMIN_ENABLEDbroadcastandhassendSMSMessagepermission

Sta*s*calfeatures• Countcer*ficatefieldsinsamplestoget100stringswithdiscrimina*[email protected]/benign=52

205 3 34.5 143234

285 68 296 7

13850 157 11218 847

1.23e+9 422 1004 177

0 398 13.333 125

Numeraliza*on(N=1235)

Con*nuousvalue（N=571）

0-1value（N=664）

0 0 0 0

0 0 0 1

0 1 0 0

0 1 1 0

0 1 0 0

‘’’’’’’’’’’’’’

‘’’’’’’’’’’‘

Page 8: AI Based An$virus: Detec$ng Android Malware Variants With a

CuVngtechnique

Feature normalization

Con*nuousvalue

Gaussiandistribu*on

[-1,1]

Standardscorenormaliza*on

Tomakefeaturesmorediscrimina*vePrecisionincreasedby9%

Noiseproblem

Long-taileddistribu*on

Quan*lenormaliza*on

CuVngtechnique

Mul*modaldistribu*on

Page 9: AI Based An$virus: Detec$ng Android Malware Variants With a

Tanh ReLU

Training in deep neural network

AutoEncoder

Configura*ons:• Hiddenlayerac*va*onfunc*on:TanhandReLU• Costfunc*on:Mul*classcrossentropy• Learningmethod:ADADELTA• Finallayerac*va*onfunc*on:Sormax• Passes:20–30

Inputlayer

NormalizedCon*nuousvalue(n=571)

0-1value(n=664)

Hiddenlayer1

Hiddenlayer2 Hiddenlayer3 Residuallayer iden*ty

outputlayer

n=256

n=256

ReLU

ReLU

Tanh

n=256 n=256

Sormax

n=256

Tanh Tanh

NetworkArchitecture

0

1

TrainedonPaddlePaddleplaYormwith15M+samples

Page 10: AI Based An$virus: Detec$ng Android Malware Variants With a

Prediction & Evaluation

Models

ExtractapkfeaturesonthephonePerf:140ms/apkTraffic:1kB/apk

Sendfeaturestothecloud

Returnpredic*ontothephone

Predic*on

Predictinthecloud

Apkfeatures

Produc+ondeployment

0.880.890.90.910.920.930.940.950.960.97

Jan2016 Mar2016 May2016 Jul2016

Thelife*meofmodeltrainedonJan2016

ThemodelistrainedonJan2016andtestedagainstAV-TESTJan,Mar,MayandJuly’ssamples.Recallratedroppedby7.6%in6months.

0.95

0.955

0.96

0.965

0.97

0.975

0.98

0.985

0.99

0.995

0 0.02 0.04 0.06 0.08 0.1

Detec*onperformanceasROCcurve

ROCcurveistestagainstAV-TESTJuly’ssamples:7613Androidmalware,3020legi*mateAndroidapps,total10633.

Detec+onperformance

True

Posi*veRate

FalsePosi*veRate

Modellife+me

Recall

Page 11: AI Based An$virus: Detec$ng Android Malware Variants With a

Limitations • Can’t provide explanations

for its detection results • Can’t understand code

meaning. • Build on static analysis and

lack of dynamic inspection. • Can’t self learning, need

continuous training with labeled data.

Advantages • More difficult to evade • Fixed-size

Page 12: AI Based An$virus: Detec$ng Android Malware Variants With a

Conclusion • Feature extraction is the key step

•  Virus analyst experience can help to find valuable features. •  AutoEncoder neural network can be used to extract the most valuable

features from a large number of features. • This system is designed to detect Android malware, but these

methods can also be used in detecting malware in other platforms.

• Our system learns in image recognition way. It’s effective only in detecting malware variants.

Page 13: AI Based An$virus: Detec$ng Android Malware Variants With a

Thank you • Welcome contact me

•  Twitter: @thomaslwang •  Email: [email protected]

• Welcome cooperation and partnership with us • Acknowledgement

•  Baidu IDL: Lyv Qin, Xiao Zhou, Jie Zhou, Errui Ding, Yuanqing Lin, Andrew Ng

•  Partner: Liuping Hou, Jinke Liu, Zhijun Jia, Yanyan Ji •  PaddlePaddle platform http://paddlepaddle.org


Document Title illwerke vkw - DETEC

Document Title illwerke vkw - DETEC

Documents
Intrusion Detection and Malware Analysis - Malware collection134.2.173.140/lehre/ws10/ids-malware/12-malware-collection.pdf · Intrusion Detection and Malware Analysis Malware collection

Intrusion Detection and Malware Analysis - Malware collection134.2.173.140/lehre/ws10/ids-malware/12-malware-collection.pdf · Intrusion Detection and Malware Analysis Malware collection

Documents
A Look at DNS-Changing Malware · 2012-01-11 · KOOBFACE variants, which rampantly spread via social networking sites, also had DNS-changing malware as possible payloads. In this

A Look at DNS-Changing Malware · 2012-01-11 · KOOBFACE variants, which rampantly spread via social networking sites, also had DNS-changing malware as possible payloads. In this

Documents
Malware Analysis and Antivirus Technologies: Kernel Malware & A Look at Malware … · 2011-08-16 · Malware Analysis and Antivirus Technologies: Kernel Malware & A Look at Malware

Malware Analysis and Antivirus Technologies: Kernel Malware & A Look at Malware … · 2011-08-16 · Malware Analysis and Antivirus Technologies: Kernel Malware & A Look at Malware

Documents
Malware Analysis Analysis.pdf · Definisi Malware Analysis • What is a malware? • Malware (malicious software) ... Malware Sinkronisasi Token • Pelaku: Zeus Banking Trojan •

Malware Analysis Analysis.pdf · Definisi Malware Analysis • What is a malware? • Malware (malicious software) ... Malware Sinkronisasi Token • Pelaku: Zeus Banking Trojan •

Documents
Malware Today and Mail Server Security - Trend Micro€¦ · Trend Micro detected more than 40 variants of NUWAR in a two-week period in late 2006. These variants present different

Malware Today and Mail Server Security - Trend Micro€¦ · Trend Micro detected more than 40 variants of NUWAR in a two-week period in late 2006. These variants present different

Documents
Ueda2015 early detec of dm dr.ihab salem

Ueda2015 early detec of dm dr.ihab salem

Documents
Semantic Trace-based Malware Variants Detectioncrest.cs.ucl.ac.uk/cow/12/slides/Khalidcow12.pdf · Semantic malware detection. Technical Report TR-10-03, Department of Computer Science,

Semantic Trace-based Malware Variants Detectioncrest.cs.ucl.ac.uk/cow/12/slides/Khalidcow12.pdf · Semantic malware detection. Technical Report TR-10-03, Department of Computer Science,

Documents
deTec HG, 8023148 - SICK

deTec HG, 8023148 - SICK

Documents
SlidesbyJeffreyD.Ullman · Highdim. data Locality) sensive) hashing) Clustering) Dimensional ity) reducon) Graph’’ data PageRank,) SimRank) Community) Detec-on) Spam) Detec-on)

SlidesbyJeffreyD.Ullman · Highdim. data Locality) sensive) hashing) Clustering) Dimensional ity) reducon) Graph’’ data PageRank,) SimRank) Community) Detec-on) Spam) Detec-on)

Documents
Malware Detection in Embedded Systems using Neural Network ... · power, and cost. Thus, existing internal malware detec-tion techniques (e.g. hardware support [47], dynamic analysis

Malware Detection in Embedded Systems using Neural Network ... · power, and cost. Thus, existing internal malware detec-tion techniques (e.g. hardware support [47], dynamic analysis

Documents
Detection of Metamorphic Malware Variants Using Global Control Flow Analysis

Detection of Metamorphic Malware Variants Using Global Control Flow Analysis

Documents
G DATA PC Malware Report H2 2015 EN · PDF fileG DATA PC Malware Report 3 ¡ In the second half of 2015, experts at G DATA SecurityLabs recorded 2,098,062 new signature variants. This

G DATA PC Malware Report H2 2015 EN · PDF fileG DATA PC Malware Report 3 ¡ In the second half of 2015, experts at G DATA SecurityLabs recorded 2,098,062 new signature variants. This

Documents
€¦ · supply chain new increase in ansomware mobile malware variants attacks variants cloudnexus Contact us about our Network Analysis Service info@cloudnexusit.com | (502)440-1380

€¦ · supply chain new increase in ansomware mobile malware variants attacks variants cloudnexus Contact us about our Network Analysis Service [email protected] | (502)440-1380

Documents
Optimized Invariant Representation of Network Traffic for ... · Optimized Invariant Representation of Network Traffic for Detecting Unseen Malware Variants ... approach is to transform

Optimized Invariant Representation of Network Traffic for ... · Optimized Invariant Representation of Network Traffic for Detecting Unseen Malware Variants ... approach is to transform

Documents
Malware Fails Best Bugs in Malware Felix Leder [Malware ... · 1 Malware Fails Best Bugs in Malware Felix Leder [Malware Detection Team] Felix.Leder@norman.com 5. desember 2011 malware

Malware Fails Best Bugs in Malware Felix Leder [Malware ... · 1 Malware Fails Best Bugs in Malware Felix Leder [Malware Detection Team] [email protected] 5. desember 2011 malware

Documents
Exploiting Similarity Between Variants to Defeat Malware€¦ · 04/01/2007 | Blackhat DC | Walenstein Exploiting Similarity Between Variants 5 So Few Families, So Many Variants Clearly

Exploiting Similarity Between Variants to Defeat Malware€¦ · 04/01/2007 | Blackhat DC | Walenstein Exploiting Similarity Between Variants 5 So Few Families, So Many Variants Clearly

Documents
evaluation of electrochemical impedance techniques for detec

evaluation of electrochemical impedance techniques for detec

Documents
metal gas detec robo

metal gas detec robo

Documents
G DATA PC Malware Report H2 2015 EN...G DATA PC Malware Report 3 ¡ In the second half of 2015, experts at G DATA SecurityLabs recorded 2,098,062 new signature variants. This number

G DATA PC Malware Report H2 2015 EN...G DATA PC Malware Report 3 ¡ In the second half of 2015, experts at G DATA SecurityLabs recorded 2,098,062 new signature variants. This number

Documents
Unsupervised Anomaly-based Malware Detection using ...simha/preprint_raid14.pdf · isting malware detection techniques can be classi ed along two dimensions: detec-tion approach and

Unsupervised Anomaly-based Malware Detection using ...simha/preprint_raid14.pdf · isting malware detection techniques can be classi ed along two dimensions: detec-tion approach and

Documents
-[ OS X Malware ]- - PUT.AS...Starting point: Macs are immune to malware. ! Latest Flashback variants broke that myth. ! In fact, it’s quite easy to write high quality OS X malware!

-[ OS X Malware ]- - PUT.AS...Starting point: Macs are immune to malware. ! Latest Flashback variants broke that myth. ! In fact, it’s quite easy to write high quality OS X malware!

Documents
Fire Related Crime Investigation and Detec. Course

Fire Related Crime Investigation and Detec. Course

Documents
INTERNET SECURITY THREAT REPORT …...83 Top-Ten Types of Information Exposed 87 E-CRIME & MALWARE 89 Value of Information Sold on Black Market 90 New Malware Variants 90 Email Malware

INTERNET SECURITY THREAT REPORT …...83 Top-Ten Types of Information Exposed 87 E-CRIME & MALWARE 89 Value of Information Sold on Black Market 90 New Malware Variants 90 Email Malware

Documents
DETEC¸ AO DE MALWARE DE NOVA GERAC¸˜ AO E˜ SUA MITIGAC ...£… · tra malware (malicious software ou software malicioso) de ultima gerac¸´ ˜ao, considerando os danos que estes,

DETEC¸ AO DE MALWARE DE NOVA GERAC¸˜ AO E˜ SUA MITIGAC ...£… · tra malware (malicious software ou software malicioso) de ultima gerac¸´ ˜ao, considerando os danos que estes,

Documents
Perfecting Your Defensive Strategy Playbook · 2018-10-03 · New malware variants are created by hackers every hour. In fact, AV-Test reports that more than 390,000 new malware variants

Perfecting Your Defensive Strategy Playbook · 2018-10-03 · New malware variants are created by hackers every hour. In fact, AV-Test reports that more than 390,000 new malware variants

Documents
PANDEMIYA EMERGES AS NEW MALWARE ALTERNATIVE · PDF filePANDEMIYA EMERGES AS NEW MALWARE ALTERNATIVE TO ZEUS-BASED VARIANTS. ... MAY 2014 ©2014 EMC Corporation. EMC, RSA, ... RSA

PANDEMIYA EMERGES AS NEW MALWARE ALTERNATIVE · PDF filePANDEMIYA EMERGES AS NEW MALWARE ALTERNATIVE TO ZEUS-BASED VARIANTS. ... MAY 2014 ©2014 EMC Corporation. EMC, RSA, ... RSA

Documents
Lecture9 Detec Tors

Lecture9 Detec Tors

Documents
Malware Detection in Embedded Systems using Neural …power, and cost. Thus, existing internal malware detec-tion techniques (e.g. hardware support [47], dynamic analysis [46], etc.)

Malware Detection in Embedded Systems using Neural …power, and cost. Thus, existing internal malware detec-tion techniques (e.g. hardware support [47], dynamic analysis [46], etc.)

Documents
Cyber 2020 PitchDeck 091217 - eecis.udel.eduOver 100K malware variants created every hour *Bad actors have embraced automation *Good actors still construct malware detection rules

Cyber 2020 PitchDeck 091217 - eecis.udel.eduOver 100K malware variants created every hour *Bad actors have embraced automation *Good actors still construct malware detection rules

Documents