Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Lecture4:3DReconstruction
CSE152:ComputerVisionManmohanChandraker
Quiz
Gradients1(a) ForimageI,areIx andIy smallorlargeatpointsA,B,CandD?
x
y
A B
C
D
Corners2(a) Isthissecondmomentmatrixlikelytorepresentacorner?Why?
2(b) Isthissecondmomentmatrixlikelytorepresentacorner?Why?
2(c) Isthissecondmomentmatrixlikelytorepresentacorner?Why?
SIFT3(a)WhatareinvariancesencodedbytheSIFTfeature?
3(b) Youobtained10truepositives,5falsepositivesand5falsenegativesataparticularSIFTmatchingthreshold.
Howwouldyouchangethethresholdto:(i) Increasetruepositives?(ii) Decreasefalsepositives?(iii) Decreasefalsenegatives?
Recap
Basicidea:• Take16x16squarewindowarounddetectedfeature• Computeedgeorientation (angleofthegradient- 90°)foreachpixel• Throwoutweakedges(threshold gradientmagnitude)• Createhistogramofsurviving edgeorientations
ScaleInvariantFeatureTransform
Adaptedfromslide byDavidLowe
0 2p
angle histogram
Modelingprojection
• Thecoordinatesystem– Wewillusethepinholemodelasanapproximation– Puttheopticalcenter(CenterOfProjection)attheorigin– Puttheimageplane(ProjectionPlane)infrontoftheCOP– Thecameralooksdownthenegativezaxis.
Modelingprojection
• Projectionequations– ComputeintersectionwithPPofrayfrom(x,y,z)toCOP– Derivedusingsimilartriangles
– Wegettheprojectionbythrowingoutthelastcoordinate:
Homogeneous coordinatesIs this a linear transformation?
Trick: add one more coordinate:
homogeneous image coordinates
homogeneous scene coordinates
Converting from homogeneous coordinates
• No, since division by z is nonlinear
Modelingprojection
• Amatrixmultiplicationusinghomogeneouscoordinates
Perspectiveprojection
(intrinsics) (convertsfrom3Draystopointsonimageplane)
Projection
dd
Fromimageplanetopixelcoordinates
(intrinsics)Projection
(0,0)
(-cx,-cy)
dd
d
(convertsfrom3Draystopointsonimageplane)
Fromimageplanetopixelcoordinates
(intrinsics)
Ingeneral,
:aspectratio(1unlesspixelsarenotsquare)
:skew (0unlesspixelsareshapedlikerhombi/parallelograms)
:principalpoint ((0,0) unlessopticalaxisdoesn’t intersectprojectionplaneatorigin)
(upper triangularmatrix)
(convertsfrom3Draysincameracoordinatesystemtopixelcoordinates)
Projection
dd
dd
Cameraparameters
• Howcanwemathematicallydescribeacamera?
• Weneedtodescribeitsinternalparameters• Weneedtodescribeitsposeintheworld
ATaleofTwoCoordinateSystems
“TheWorld”
Camera
x
y
z
v
w
u
o
COP
Twoimportantcoordinatesystems:1.Worldcoordinatesystem2.Cameracoordinatesystem
Cameraparameters• Toprojectapoint(x,y,z)inworld coordinatesintoacamera
• Firsttransform(x,y,z)intocamera coordinates• Needtoknow
– Cameraposition(inworldcoordinates)– Cameraorientation(inworldcoordinates)
• Thenprojectintotheimageplane– Needtoknowcameraintrinsics
• Thesecanallbedescribedwithmatrices.
Extrinsics• Howdowegetthecamerato“canonicalform”?
– (Centerofprojectionattheorigin,x-axispointsright,y-axispointsup,z-axispointsbackwards)
0
Step1:Translateby-c
Extrinsics• Howdowegetthecamerato“canonicalform”?
– (Centerofprojectionattheorigin,x-axispointsright,y-axispointsup,z-axispointsbackwards)
0
Step1:Translateby-c
Howdowerepresenttranslationasamatrixmultiplication?
Extrinsics• Howdowegetthecamerato“canonicalform”?
– (Centerofprojectionattheorigin,x-axispointsright,y-axispointsup,z-axispointsbackwards)
0
Step1:Translateby-cStep2:RotatebyR
3x3rotationmatrix
Extrinsics• Howdowegetthecamerato“canonicalform”?
– (Centerofprojectionattheorigin,x-axispointsright,y-axispointsup,z-axispointsbackwards)
0
Step1:Translateby-cStep2:RotatebyR
Projectionmatrix
translationrotationprojectionintrinsics
Denotethisbyt.
Projectionmatrix
0
=
(inhomogeneous imagecoordinates)
StructurefromMotion(SFM)VisualSLAM
FeaturedetectionSeveralimagesobserveascenefromdifferentviewpoints
FeaturedetectionDetectfeaturesusing,forexample,SIFT[Lowe,IJCV2004]
FeaturematchingMatchfeaturesbetweeneachpairofimages
Structurefrommotion
Camera1
Camera2
Camera3R1,t1
R2,t2R3,t3
X1
X4X3
X2
X5X6
X7
minimizeg(R,T,X)
p1,1p1,2
p1,3
non-linearleastsquares
Bundleadjustment• Minimizesumofsquaredreprojection errors:
– Optimizedwithnon-linearleastsquares– Levenberg-Marquardtisapopularchoice
• Practicalchallenges?– Initialization– Outliers
predictedimagelocation
observedimagelocationindicatorvariable:
whetherpoint i visibleinimagej
Featurematching
Robustness• Let’sconsiderasimplerexample…linefitting
• Howcanwefixthis?Problem:Fitalinetothesedatapoints Leastsquaresfit
Idea
• Givenahypothesizedline• Countthenumberofpointsthat“agree” withtheline– “Agree” =withinasmalldistanceoftheline– Thesearetheinlierstothatline
• Forallpossiblelines,selecttheonewiththelargestnumberofinliers
Countinginliers
Countinginliers
Inliers:3
Countinginliers
Inliers:20
Howdowefindthebestline?
• Unlikeleast-squares,nosimpleclosed-formsolution
• Hypothesize-and-test– Tryoutmanylines,keepthebestone– RANSAC:RandomSampleConsensus
• Numberofsamplesdependson– Outlierratio– Probabilityofcorrectanswer– Modelsize
RANSAC
• Generalversion:1. Randomlychooses samples
• Typicallys =minimumsamplesizetofitamodel
2. Fitamodel(say,line)tothosesamples3. Countthenumberofinliersthatapproximatelyfit
themodel4. RepeatN times5. Choosethemodelwiththelargestsetofinliers
Two-ViewReconstruction
Image1
Image2
R1,t1
R2,t2
X
FundamentalMatrix
x1x2
xT1Fx2 = 0
x1↔ x2
RANSACtoEstimateFundamentalMatrix
• Formanytimes– Pick8points– Computeasolutionforusingthese8points– Countnumberofinlierswithcloseto0
• Picktheonewiththelargestnumberofinliers
FxT1Fx2