Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,

AlgorithmicIntelligenceLab


EE807:RecentAdvancesinDeepLearningLecture13

Slidemadeby

Kimin LeeKAISTEE

NoveltyDetectionforDeepClassifiers


1. Introduction• Whatisnoveltydetection?• Overview

2. UtilizingthePosteriorDistribution• Baselinemethod• Post-processingmethod

3. UtilizingtheHiddenFeatures• Localintrinsicdimensionality• Mahalanobis distance-basedscore

TableofContents

2





TableofContents

3


• Deepneuralnetworks(DNNs)canbegeneralizedwell whenthetestsamplesarefromsimilardistribution(i.e.,in-distribution)

WhatisNoveltyDetection?

4

Trainingdata=animal

TestsampleDNNs

Softmax

cat dog

0.99


• Deepneuralnetworks(DNNs)canbegeneralizedwell whenthetestsamplesarefromsimilardistribution(i.e.,in-distribution)

• However,intherealworld,therearemanyunknownandunseensamples thatclassifiercan’tgivearightanswer


5

Trainingdata=animal

TestsampleDNNs

Softmax

cat dog

0.99

Unseensample,i.e.,out-of-distribution(notanimal)

Unknownsample Adversarialsamples[Goodfellow etal.,2015]


• Noveltydetection• Givenpre-trained(deep)classifier,• Detectwhetheratestsampleisfromin-distribution(i.e.,trainingdistributionbyclassifier)ornot(e.g.,out-of-distribution/adversarialsamples)


6

Decisionboundary

Abnormalsample



• Itcanbeusefulformanymachinelearningproblems:


7

Decisionboundary

Abnormalsample

Calibration[Guoetal.,2017]

Ensemblelearning[Leeetal.,2017]

Incrementallearning[Rebuff etal.,2017]



• ItisalsoindispensablewhendeployingDNNsinreal-worldsystems [Amodei etal.,2016]


8

Decisionboundary

Abnormalsample

Autonomousdrive Secureauthenticationsystem


• Howtosolvethisproblem?• Threshold-basedDetector[Hendrycks etal.,2017,Liangetal.,2018]


9

[Testsample] [DeepClassifier]

score10

Ifscore>𝜖:In-distribution

Else:out-of-distribution




10


score10



Howtogetconfidencescore



• Utilizingaposterior distribution• 1.Maximumvalueorentropyofposterior[Hendrycks etal.,2017]

• 2.Inputandoutputprocessing[Liangetal.,2018]

• 3.Bayesianinference[Lietal.,2017]andensembleofclassifier[Balajietal.,2017]


11


Softmax

Persiancat

tigercat

0.120.18



• UtilizingahiddenfeaturesfromDNNs• 1.Localintrinsicdimensionality[Maetal.,2018]

• 2.Mahalanobis distance[Leeetal.,2018b]


12


Softmax

Persiancat

tigercat

0.120.18





TableofContents

13


• Remindthatclassificationisfindinganunknownposteriordistribution,i.e.,P(Y|X)

• Howtomodelourposteriordistribution:Softmax classifierwithDNNs

• WhereishiddenfeaturesfromDNNs

UtilizingthePosteriorDistribution

14

Inputspace Outputspace


• Remindthatclassificationisfindinganunknownposteriordistribution,i.e.,P(Y|X)

• Howtomodelourposteriordistribution:Softmax classifierwithDNNs

• WhereishiddenfeaturesfromDNNs

• Naturalchoiceforconfidencescore• 1.maximumvalueofposteriordistribution

• 2.entropyofposteriordistribution


15

Inputspace Outputspace


• Baselinedetector[Hendrycks etal.,2017]• Confidencescore=maximumvalueofpredictivedistribution


16

[Input] [Deepclassifier]





• Evaluation:detectingout-of-distribution• AssumethatwehaveclassifiertrainedonMNISTdataset• Detectingout-of-distributionforthisclassifier


17




In-distribution Out-of-distribution

Predictivedist.

Data



• Evaluation:detectingout-of-distribution• TP=truepositive/FN=falsenegative/TN=truenegative/FP=falsepositive


18




• AUROC• AreaunderROCcurve• ROCcurve=relationshipbetweenTPRandFPR

• AUPR(AreaunderthePrecision-Recallcurve)• AreaunderPRcurve• PRcurve=relationshipbetweenprecisionandrecall



• Evaluation:detectingout-of-distribution• Imageclassification(computervision)


19




Baselinemethodisbetterthanrandomdetector



• Evaluation:detectingout-of-distribution• Textcategorization(NLP)

• Out-of-distribution• 5Newsgroupsfor15Newsgroups• 2ReutersforReuters6• 12Reutersfor40Reuters


20





• ODINdetector[Liangetal.,2018]• Calibratingtheposteriordistributionusingpost-processing

• Twotechniques• Temperaturescaling

• Relaxingtheoverconfidencebysmoothingtheposteriordistribution


21

Temperaturescalingparameter




• Inputpreprocessing


22

Magnitudeofnoise





• Usingtwomethods,theauthorsdefineconfidencescoreasfollows:


23





• Usingtwomethods,theauthorsdefineconfidencescoreasfollows:

• Howtoselecthyper-parameters• Validation

• 1000imagesfromin-distribution(positive)• 1000imagesfromout-of-distribution(negative)


24


• Experimentalresults


25



2. UtilizingthePosteriorDistribution• Baseline method• Post-processingmethod


TableofContents

26


• Motivation• HiddenfeaturesfromDNNscontainmeaningfulfeaturesfromtrainingdata

• Theycanbeusefulfordetectingabnormalsamples!

UtilizingtheHiddenFeatures

27

LotsofData

Objects

Edge Parts


• LocalIntrinsicDimensionality(LID)[Maetal.,2018]• Expansiondimension

• Rateofgrowthinthenumberofdataencounteredasthedistancefromthereferencesampleincreases(𝑉 isvolume)


28




• LID=expansiondimensioninthestatisticalsetting

• Where𝐹 isanalogoustothevolumeinequation(1)


29




• LID=expansiondimensioninthestatisticalsetting

• Where𝐹 isanalogoustothevolumeinequation(1)• EstimationofLID[Amsaleg etal.,2015]


30

distancebetweensampleanditsk-th nearestneighbor


• MotivationofLID• Abnormalsamplemightbescatteredcomparedtonormalsamples

• ThisimpliesthatLIDcanbeusefulfordetectingabnormalsamples!


31




• Evaluation:detectingadversarialsamples[Szegedy,etal.,2013]• Misclassifiedexamplesthatareonlyslightlydifferentfromoriginalexamples


32

*Thistopicwillbecoveredinthenextlecture




• Evaluation:detectingadversarialsamples[Szegedy,etal.,2013]


33

=


• Empiricaljustification

• Adversarialsamples(generatedbyOPTattack[Carlini etal.,2017])canbedistinguishedusingLID

• LIDsfromlow-levellayersarealsousefulindetection


34


• Mainresultsondetectingadversarialattacks• Testedmethod

• Bayesianuncertainty(BU)andDensityestimator(DE)[Feinman etal.,2017]

• LIDoutperformsallbaselinemethods


35


• Mahalanobis distance-basedconfidencescore[Leeetal.,2018]


36


• Mahalanobis distance-basedconfidencescore[Leeetal.,2018]• Givenpre-trainedSoftmax classifierwithDNNs


37



• Inducingagenerativeclassifieronhiddenfeaturespace


38

penultimate




• Motivation:connectionbetween softamx andgenerativeclassifier(LDA)


39

penultimate

~ =




• Theparametersofgenerativeclassifier=samplemeansandcovariance• Giventrainingdata


40

penultimate


• Usinggenerativeclassifier,wedefinenewconfidencescore:

• Measuringthelogoftheprobabilitydensitiesofthetestsample


41




• Intuition


42




• Boostingtheperformance• Inputpre-processing

• MotivatedbyODIN[Liangetal.,2018]


43





• Featureensemble


44

FittingGaussianusingfeaturesfromintermediatelayers





• Featureensemble

• Intuition:low-levelfeaturealsocanbeusefulfordetectingabnormalsamples


45

FittingGaussianusingfeaturesfromintermediatelayers


• Mainalgorithm

• Remarkthat• Wecombinetheconfidencescoresfrommultiplelayersusingweightedensemble

• Ensembleweightsareselectedbyutilizingthevalidationset


46


• Experimentalresultsondetectingout-of-distribution• Contributionbyeachtechnique


47

Baseline[13]:maximumvalueofposteriordistributionODIN[21]:maximumvalueofposteriordistributionafterpost-processingOurs:theproposedMahalanobis distance-basedscore


• Experimentalresultsondetectingout-of-distribution• Mainresults

• Forallcases,oursoutperformsODINandbaselinemethod• Validationconsistsof1Kdatafromeachin- andout-of-distributionpair• Validationconsistsof1Kdatafromeachin- andcorrespondingFGSMdata

• Noinformationaboutout-of-distribution


48


• Experimentalresultsondetectingadversarialattacks• Mainresults

• Foralltestedcases,ourmethodoutperformsLIDandKDestimator• Forunseenattacks,ourmethodisstillworkingwell

• FGSMsamplesdenotedby“seen”areusedforvalidation


49


• Inthislecture,wecovervariousmethodsfordetectingabnormalsampleslikeout-of-distributionandadversarialsamples• Posteriordistribution-basedmethods• Hiddenfeature-basedmethods

• Therearealsotrainingmethodsforobtainingmorecalibratedscores• Ensembleofclassifier[Balajietal.,2017]• Bayesiandeepmodels[Lietal.,2017]• CalibrationlosswithGAN[Leeetal.,2018a]

• Suchmethodscanbeusefulformanymachinelearningapplications• Activelearning[Galetal.,2017]• Incrementallearning[Rebuff etal.,2017]• Ensemblelearning[Leeetal.,2017]• Networkcalibration[Guoetal.,2017]

Summary

50


[Hendrycks etal.,2017]Abaselinefordetectingmisclassifiedandout-of-distributionexamplesinneuralnetworks. InICLR2017.https://arxiv.org/abs/1610.02136

[Maetal.,2018]CharacterizingAdversarialSubspacesUsingLocalIntrinsicDimensionality. InICLR,2018.https://openreview.net/pdf?id=B1gJ1L2aW

[Feinman etal.,2017]Detectingadversarialsamplesfromartifacts. arXiv preprintarXiv:1703.00410,2017.https://arxiv.org/abs/1703.00410

[Lee,etal.,2018a]TrainingConfidence-calibratedClassifiersforDetectingOut-of-DistributionSamples,InICLR,2018.https://arxiv.org/abs/1711.09325

[Lee,etal.,2018b]ASimpleUnifiedFrameworkforDetectingOut-of-DistributionSamplesandAdversarialAttacks,InNIPS,2018.https://arxiv.org/abs/1807.03888

[Liang,etal.,2018]PrincipledDetectionofOut-of-DistributionExamplesinNeuralNetworks. InICLR,2018.https://arxiv.org/abs/1706.02690

[Goodfellow etal.,2015]Explainingandharnessingadversarialexamples. InICLR,2015.https://arxiv.org/pdf/1412.6572.pdf

[Amodei,etal.,2016]Concreteproblemsinai safety.arXiv preprintarXiv:1606.06565,2016.https://arxiv.org/abs/1606.06565

[Guoetal.,2017]OnCalibrationofModernNeuralNetworks. InICML,2017.https://arxiv.org/abs/1706.04599

[Leeetal.,2017]ConfidentMultipleChoiceLearning.InICML,2017.https://arxiv.org/abs/1706.03475

[Balajietal.,2017]SimpleandScalablePredictiveUncertaintyEstimationusingDeepEnsembles,InNIPS,2017.https://arxiv.org/pdf/1612.01474.pdf

References

51


[Rebuff etal.,2017]iCaRL:IncrementalClassifierandRepresentationLearning. InCVPR,2017.https://arxiv.org/pdf/1611.07725.pdf

[Huangetal.,2017]Denselyconnectedconvolutionalnetworks,InCVPR,2017.https://arxiv.org/abs/1608.06993

[Zagoruyko etal.,2016]Wideresidualnetworks,InBMVC2016.https://arxiv.org/pdf/1605.07146.pdf

[Amsaleg etal.,2015]Estimatinglocalintrinsicdimensionality.InSIGKDD,2015.http://mistis.inrialpes.fr/~girard/Fichiers/p29-amsaleg.pdf

[Szegedy etal.,2013]Intriguingpropertiesofneuralnetworks. arXiv preprintarXiv:1312.6199,2013.https://arxiv.org/abs/1312.6199

[Lietal.,2017]DropoutInferenceinBayesianNeuralNetworkswithAlpha-divergences,InICML,2017.https://arxiv.org/abs/1703.02914

[Galetal.,2017]DeepBayesianActiveLearningwithImageData,InICML,2017.https://arxiv.org/abs/1703.02910

[Carlini etal.,2017]Towardsevaluatingtherobustnessofneuralnetworks.In IEEESP,2017.https://arxiv.org/abs/1608.04644

References

52

Documents

Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,