Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
WhatUncertaintiesDoWeNeedinBayesianDeepLearningforComputerVision?AlexKendall([email protected])andYarinGal([email protected])
InBayesianmodelling,therearetwomaintypesofuncertaintywecanmodel[1]:• Epistemicuncertainty:uncertaintyinthemodel,capturingwhatourmodeldoesn’tknowduetolackoftrainingdata.Canbeexplainedawaywithincreasedtrainingdata.
• Aleatoricuncertainty:informationwhichourdatacannotexplain.Canbeexplainedawaywithincreasedsensorprecision.
Itisimportanttomodelaleatoric uncertaintyfor:• Largedatasituations,whereepistemicuncertaintyisexplainedaway,• Real-timeapplications,becausewecanformaleatoricmodelswithoutexpensiveMCsamples.
• Noisydata,becausewecanlearntoattenuateerroneouslabels.
withdeeplearning.Ourmodel’suncertaintyforpixeloutput𝑦" isgivenby:
𝑉𝑎𝑟 𝑦" ≈1𝑇)𝜎 𝑥, -
�
/
+1𝑇)𝑓 𝑥, -
�
/
−1𝑇)𝑓 𝑥,
�
/
-
UsingMonteCarlodropoutsamples,T,learningaleatoricuncertaintywithloss:
𝐿𝑜𝑠𝑠 𝜃 =1𝐷)
12𝜎 𝑥 "
�
"
𝑦" − 𝑓(𝑥)"- + log𝜎 𝑥 "
InputImageGroundTruthModelSegmentationAleatoricUncertaintyEpistemicUncertainty
Experimentstrainingononedatasetandtestingonanother.• Aleatoricuncertaintycannotbeexplainedawaywithmoredata,
• Aleatoricuncertaintydoesnotincreaseforout-of-dataexamples(situationsdifferentfromtrainingset),
• Epistemicuncertaintyincreaseswithdecreasingtrainingsize,
• Epistemicuncertaintyincreaseswithexamplesoutofthetrainingdistribution.
forsemanticsegmentationandper-pixeldepthregressiondatasets.
WeuseaconvolutionalnetworkbasedonDenseNet [20]with103layersand9.4Mparameters
Per-pixeldepthregression
Modellinguncertaintyallowsthemodeltolearntoattenuatetheeffectfromerroneouslabelsandlearnlossattenuation.
InputImage DepthRegressionUncertainty
Andepistemic uncertaintyisimportantfor:• Safety-criticalapplications,becauseepistemicuncertaintyisrequiredtounderstandexampleswhicharedifferentfromtrainingdata,
• Smalldatasetswherethetrainingdataissparse.
1.TypesofUncertainty 2.Wejointlymodelaleatoric andepistemicuncertainty 3.SOTAperformance
4.UncertaintywithDistancefromTrainingData
5.Conclusions