Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
DiscreteInferenceandLearningLecture4
MVA2020–2021
http://thoth.inrialpes.fr/~alahari/disinflearn
SlidesbasedonmaterialfromNikosKomodakis,M.PawanKumar
Outline
• Previousclasses– Graphcuts,Beliefpropagationandvariants– (Inference)
• Today– Quickrecapofthecourse– Learningparameters
Beforemovingon…
Projectsuggestions(alsosentbyemail)
• ImplementBPontrees,thengraph,extendtoTRW,compare• Implementgraphcut+extension(Ishikawa,othermulti-label)or
variationofimplementation+smallapplication• Complexapplicationofgraphcut,requiringmodelling(e.g.,
sequenceofimages)• Geometricscenelabellingwithgraphcuts• Jointmodellingoftwolabellingproblems(e.g.,segmentation+
detection)• Implementfastprimal-dualalgorithm+evaluate• Implementdeformablepartsmodelforobjectdetection• …• Oryourown(butcheckwithusfirst)• Selectprojectsbefore25thJanuaryandemailus
Projects
• Chooseprojectsbefore25/1(Monday!)
• Presentationson31/3– InEnglishorFrench– 15min,includingquestions
• Reportdueon30/3
Recap
• Whatinferencealgorithmwouldyouusefor– agraphwithonlychains• 2-labelproblem?• Multi-labelproblem?
– Treestructuredgraph• 2labelproblem?• Multi-labelproblem?
Recap• Basics:problemformulation– EnergyFunction– MAPEstimation– Computingmin-marginals– Reparameterization
• Solutions– BeliefPropagationandrelatedmethods[Lecture3]– Graphcuts[Lecture2]
Outline
• Recapofthecourse
• Learningparameters
ConditionalRandomFields(CRFs)• Ubiquitousincomputervision• segmentation stereomatchingopticalflow imagerestorationimagecompletion objectdetection/localization...
• andbeyond• medicalimaging,computergraphics,digitalcommunications,physics…
• Reallypowerfulformulation
ConditionalRandomFields(CRFs)
• Extensiveresearchformorethan20years
• Keytask:inference/optimizationforCRFs/MRFs
• Lotsofprogress
• Graph-cutbasedalgorithms• Message-passingmethods• LPrelaxations• DualDecomposition• ….
• Manystate-of-the-artmethods:
MAPinferenceforCRFs/MRFs
• Hypergraph– Nodes– Hyperedges/cliques
• High-orderMRFenergyminimizationproblem
high-orderpotential(oneperclique)
unarypotential(onepernode)
hyperedges
nodes
CRFtraining• ButhowdowechoosetheCRFpotentials?
• Throughtraining• Parameterizepotentialsbyw
• Usetrainingdatatolearncorrectw
• Characteristicexampleofstructuredoutputlearning[Taskar],[Tsochantaridis,Joachims]
• Equally,ifnotmore,importantthanMAPinference• Betteroptimizecorrectenergy(evenapproximately)• Thanoptimizewrongenergyexactly
• SupervisedLearning
• ProbabilisticMethods
• Loss-basedMethods
• Results
Outline
ImageClassification
Isthisanurbanorruralarea?
Input:d Output:x∈{-1,+1}
ImageClassification
Isthisscanhealthyorunhealthy?
Input:d Output:x∈{-1,+1}
ImageClassification
X
d
LabelingX=x LabelsetL={-1,+1}
ImageClassification
Whichcityisthis?
Input:d Output:x∈{1,2,…,h}
ImageClassification
Whattypeoftumordoesthisscancontain?
Input:d Output:x∈{1,2,…,h}
ObjectDetection
Whereistheobjectintheimage?
Input:d Output:x∈{Pixels}
ObjectDetection
Whereistheruptureinthescan?
Input:d Output:x∈{Pixels}
ObjectDetection
X
d
LabelingX=x LabelsetL={1,2,…,h}
Segmentation
Whatisthesemanticclassofeachpixel?
Input:d Output:x∈{1,2,…,h}|Pixels|
car
roadgrass
treesky
sky
Segmentation
Whatisthemusclegroupofeachpixel?
Input:d Output:x∈{1,2,…,h}|Pixels|
Segmentation
X1
d1
X2
d2
X3
d3
X4
d4
X5
d5
X6
d6
X7
d7
X8
d8
X9
d9
LabelingX=x LabelsetL={1,2,…,h}
Segmentation
X1
d1
X2
d2
X3
d3
X4
d4
X5
d5
X6
d6
X7
d7
X8
d8
X9
d9
LabelingX=x LabelsetL={1,2,…,h}
CRFtraining• Stereomatching:• Z:left,rightimage• X:disparitymap
Z X
f :
argf = parameterizedbyw
Goaloftraining:estimateproper
w
CRFtraining• Denoising:• Z:noisyinputimage• X:denoisedoutputimage
Z X
f :
argf = parameterizedbyw
Goaloftraining:estimateproper
w
CRFtraining• Objectdetection:• Z:inputimage• X:positionofobjectparts
Z X
f :
argf = parameterizedbyw
Goaloftraining:estimateproper
w
CRFtraining(somefurthernotation)
vectorvaluedfeaturefunctions
Learningformulations
Riskminimization
Ktrainingsamples
RegularizedRiskminimization
RegularizedRiskminimization
ReplaceΔ(.)witheasiertohandleupperboundLG(e.g.,convexw.r.t.w)
Choice1:Hingeloss
§ UpperboundsΔ(.)
§ Leadstomax-marginlearning
Max-marginlearning
energyofgroundtruth
anyotherenergy
desiredmargin
slack
Max-marginlearning
subjecttotheconstraints:
energyofgroundtruth
anyotherenergy
desiredmargin
slack
Max-marginlearning
subjecttotheconstraints:
energyofgroundtruth
anyotherenergy
desiredmargin
slack
Max-marginlearning
subjecttotheconstraints:
orequivalently
CONSTRAINED
UNCONSTRAINED
Choice2:logisticloss
§ Canbeshowntoleadtomaximumlikelihoodlearning
partitionfunction
Max-marginvsMaximum-likelihoodmax-margin
maximumlikelihood
Max-marginvsMaximum-likelihoodmax-margin
maximumlikelihood
soft-max
Solvingthelearningformulations
Maximum-likelihoodlearning
§ Differentiable&convex
partitionfunction
§ Globaloptimumviagradientdescent,forexample
Maximum-likelihoodlearning
gradient
Recallthat:
Maximum-likelihoodlearning
gradient
§ RequiresMRFprobabilisticinference§ NP-hard(exponentiallymanyx):approximationvialoopy-BP?
Max-marginlearning(UNCONSTRAINED)
§ Convexbutnon-differentiable§ Globaloptimumviasubgradientmethod
Subgradient
x2
subgradientatx1
g(x2)+h2·(x-x2)
subgradientatx2=gradientatx2
Subgradient
Subgradient
x
Subgradient
subgradientofLG =
Max-marginlearning(UNCONSTRAINED)
totalsubgr. =
Repeat1.computeglobalminimizersatcurrentw 2.computetotalsubgradientatcurrentw3.updatew bytakingastepinthenegativetotalsubgradient direction
untilconvergence
Subgradientalgorithm
Max-marginlearning(UNCONSTRAINED)
partialsubgradient=
Repeat1.pickkatrandom2.computeglobalminimizeratcurrentw 3.computepartialsubgradientatcurrentw4.updatew bytakingastepinthenegativepartialsubgradient direction
untilconvergence
Stochasticsubgradientalgorithm
MRF-MAPestimationperiteration(unfortunatelyNP-hard)
Max-marginlearning(CONSTRAINED)
subjecttotheconstraints:
Max-marginlearning(CONSTRAINED)
subjecttotheconstraints:
linearinw
• Quadraticprogram(great!)• Butexponentiallymanyconstraints(notsogreat)
• Whatifweuseonlyasmallnumberofconstraints?
• ResultingQPcanbesolved• Butsolutionmaybeinfeasible
Max-marginlearning(CONSTRAINED)
• onlyfewconstraintsactiveatoptimalsolution!!(variablesmuchfewerthanconstraints)
• Constraintgenerationtotherescue
• Giventheactiveconstraints,restcanbeignored• Thenletustrytofindthem!
1.Startwithsomeconstraints
Constraintgeneration
2.SolveQP
3.Checkifsolutionisfeasiblew.r.t.toallconstraints
4.Ifyes,wearedone!
5.Ifnot,pickaviolatedconstraintandaddittothecurrentsetofconstraints.Repeatfromstep2.(optionally,wecanalsoremoveinactiveconstraints)
• Keyissue:wemustalwaysbeabletofindaviolatedconstraintifoneexists
Constraintgeneration
• Recalltheconstraintsformax-marginlearning
• Tofindviolatedconstraint,wethereforeneedtocompute:
(justlikesubgradientmethod!)
1.InitializesetofconstraintsC toempty
Constraintgeneration
2.SolveQPusingcurrentconstraintsC andobtainnew(w,ξ)
4.Foreachk,ifthefollowingconstraintisviolatedthenaddittosetC:
5.Ifnonewconstraintwasaddedthenterminate.Otherwisegotostep2.
3.Computeglobalminimizersatcurrentw
MRF-MAPestimationpersample(unfortunatelyNP-hard)
Max-marginlearning(CONSTRAINED)
subjecttotheconstraints:
• Alternatively,wecansolveaboveQPinthedualdomain
• dualvariables↔primalconstraints• Toomanyvariables,butmostofthemzeroatoptimalsolution
• Useaworking-setmethod(essentiallydualtoconstraintgeneration)