SingleView Metr ology

A. Criminisi, I. ReidandA. Zisserman�

Departmentof EngineeringScienceUniversityof OxfordOxford,UK, OX1 3PJ�

criminis,ian,az� @robots.ox.ac.uk

Abstract

We describehow 3D affinemeasurementsmaybecom-putedfroma singleperspectiveview of a scenegivenonlyminimalgeometricinformationdeterminedfromtheimage.Thisminimal informationis typically the vanishingline ofa referenceplane, anda vanishingpoint for a directionnotparallel to theplane. It is shownthat affinescenestructuremaythenbedeterminedfromtheimage, withoutknowledgeof thecamera’s internal calibration(e.g. focal length),norof theexplicit relationbetweencamera andworld (pose).

In particular, we showhow to (i) computethe distancebetweenplanesparallel to thereferenceplane(upto a com-monscalefactor); (ii) computearea and lengthratios onanyplaneparallel to thereferenceplane;(iii) determinethecamera’s(viewer’s) location.Simplegeometricderivationsare givenfor theseresults. We also developan algebraicrepresentationwhich unifiesthethreetypesof measurementand,amongstotheradvantages,permitsa first order errorpropagation analysisto be performed,associatingan un-certaintywith each measurement.

We demonstrate the techniquefor a variety of applica-tions, including height measurementsin forensic imagesand3D graphicalmodellingfromsingleimages.

1. Intr oduction

In this paperwe describehow aspectsof the affine 3Dgeometryof a scenemay be measuredfrom a singleper-spective image. We will concentrateon scenescontainingplanesandparallel lines, althoughthe methodsarenot sorestricted.Themethodswe developextendandgeneralizepreviousresultsonsingleview metrology[8, 9, 13, 14].

It is assumedthat imagesare obtainedby perspectiveprojection.In addition,weassumethatthevanishingline ofa referenceplanein thescenemaybedeterminedfrom theimage,togetherwith avanishingpointfor anotherreference�

The authorswould like to thank Andrew Fitzgibbonfor assistancewith theTargetJrlibraries,andDavid Liebowitz andLuc vanGool for dis-cussions.Thiswork wassupportedby theEU EspritProjectIMPROOFS.

direction(notparallelto theplane).We arethenconcernedwith threecanonicaltypesof measurement:(i) measure-mentsof thedistancebetweenany of theplaneswhich areparallelto the referenceplane;(ii) measurementson theseplanes(andcomparisonof thesemeasurementsto thoseob-tainedonany plane);and(iii) determiningthecamera’spo-sition in termsof the referenceplaneand direction. Themeasurementmethodsdevelopedhereare independentofthecamera’s internalparameters:focal length,aspectratio,principalpoint,skew.

Theideasin thispapercanbeseenasreversingtherulesfor drawing perspective imagesgivenby LeonBattistaAl-berti [1] in his treatiseon perspective (1435). Thesearetherulesfollowedby theItalianRenaissancepaintersof the15th century, and indeedwe demonstratethe correctnessof their masteryof perspective by analysinga paintingbyPierodellaFrancesca.

Webegin in section2 bygivinggeometricinterpretationsfor the key scenefeatures,and then give simple geomet-ric derivationsof how, in principle,threedimensionalaffineinformationmay be extractedfrom the image. In section3 we introducean algebraicrepresentationof the problemandshow that this representationunifiesthe threecanoni-cal measurementtypes,leadingto simpleformulaein eachcase. In section4 we describehow errorsin imagemea-surementspropagateto errorsin the3D measurements,andhenceweareabletocomputeconfidenceintervalsonthe3Dmeasurements,i.e. a quantitative assessmentof accuracy.Thework hasavarietyof applications,andwedemonstratetwo importantones:forensicmeasurementandvirtualmod-elling in section5.

2. Geometry

The cameramodelemployedhereis centralprojection.We assumethat the vanishingline of a referenceplaneinthescenemaybecomputedfrom imagemeasurements,to-getherwith avanishingpoint for anotherdirection(notpar-allel to theplane).This informationis generallyeasilyob-tainablefrom imagesof structuredscenes[3, 11, 12]. Ef-

image plane

reference plane

camera centreplane vanishing line

vanishingpoint

v

l ref.dir.

Figure1: Basicgeometry: Theplane’s vanishingline � is theintersectionof theimageplanewith aplaneparallelto thereferenceplaneandpassingthroughthe cameracentre. The vanishingpoint � is the intersectionofthe imageplanewith a line parallelto thereferencedirectionthroughthecameracentre.

fectssuchasradialdistortion(oftenarisingin slightlywide-anglelensestypically usedin securitycameras)which cor-rupt thecentralprojectionmodelcangenerallyberemoved[6], andarethereforenot detrimentalto our methods(see,for example,figure9).

Althoughtheschematicfiguresshow thecameracentreat a finite location,the resultswe derive apply alsoto thecaseof acameracentreat infinity, i.e.wheretheimagesareobtainedby parallelprojection.Thebasicgeometryof theplane’svanishingline andthevanishingpointareillustratedin figure1. Thevanishingline � of thereferenceplaneis theprojectionof the line at infinity of thereferenceplaneintothe image.Thevanishingpoint � is the imageof thepointat infinity in thereferencedirection.Notethatthereferencedirectionneednot bevertical,althoughfor clarity we willoftenreferto thevanishingpointasthe“vertical” vanishingpoint. Thevanishingpoint is thentheimageof thevertical“footprint” of thecameracentreon thereferenceplane.

It canbe seen(for example,by inspectionof figure 1)that the vanishingline partitionsall pointsin scenespace.Any scenepoint which projectsonto the vanishingline isat the samedistancefrom the planeasthe cameracentre;if it lies “above” the line it is furtherfrom theplane,andif“below” thevanishingline, thenit is closerto theplanethanthecameracentre.

Two pointson separateplanes(parallelto the referenceplane)correspondif the line joining themis parallelto thereferencedirection;hencethe imageof eachpoint andthevanishingpoint arecollinear. For example,if thedirectionis vertical,thenthetop of anuprightperson’s headandthesoleof his/herfoot correspond.

2.1. Measurementsbetweenparallel planes

We wish to measurethe distancebetweentwo parallelplanes,specifiedby the imagepoints � and , in therefer-encedirection. Figure2 shows the geometry, with points� and in correspondence.Thefour pointsmarkedon thefiguredefineacross-ratio.Thevanishingpoint is theimageof apointat infinity in thescene[15]. In theimagethevalue

v

π

t

b

l

vanishing point

vanishing line

π /

i

Figure 2: Crossratio: The point on the plane � correspondsto thepoint � on theplane�� . They arealignedwith thevanishingpoint � . Thefour points � , � , andtheintersection� of theline joining themwith thevanishingline defineacross-ratio.Thevalueof thecross-ratiodeterminesa ratioof distancesbetweenplanesin theworld, seetext.

of thecross-ratioprovidesanaffine lengthratio. In factweobtaintheratio of thedistancebetweentheplanescontain-ing � and , to thecamera’s distancefrom theplane � (or��� dependingon theorderingof thecross-ratio).Theabso-lute distancecanbeobtainedfrom this distanceratio oncethe camera’s distancefrom � is specified. However it isusuallymorepracticalto determinethedistancevia a sec-ond measurementin the image,that of a known referencelength.

Furthermore,sincethevanishingline is theimagedaxisof the pencil of planesparallel to the referenceplane,theknowledgeof the distancebetweenany pair of the planesis sufficient to determinetheabsolutedistancebetweenan-othertwo of theplanes.Example. Figure 3 shows that a person’s heightmay becomputedfrom an imagegivena vertical referenceheightelsewherein the scene.The formulausedto computethisresultis givenin section3.1.

2.2. Measurementson parallel planes

If the referenceplane � is affine calibrated(we knowits vanishingline) thenfrom imagemeasurementswe cancompute:(i) ratiosof lengthsof parallel line segmentsonthe plane; (ii) ratiosof areason the plane. Moreover thevanishingline is sharedby the pencil of planesparalleltothereferenceplane,henceaffinemeasurementsmaybeob-tainedfor any otherplanein thepencil. However, althoughaffinemeasurements,suchasanarearatio,maybemadeonaparticularplane,theareasof regionslying on two parallelplanescannotbecompareddirectly. If theregion is parallelprojectedin thescenefrom oneplaneontotheother, affinemeasurementscanthenbemadefrom theimagesincebothregionsarenow on thesameplane,andparallelprojectionbetweenparallelplanesdoesnotalteraffineproperties.

178.8 cm

t

b

t

b

i

r

r

ir

Figure 3: Measuring the height of a person: (top) original image;(bot-tom) theheightof thepersonis computedfrom theimageas178.8cm(thetrueheightis 180cm,but notethat thepersonis leaningdown a bit on hisright foot). Thevanishingline is shown in white andthereferenceheightis thesegment( �������� ). Theverticalvanishingpoint is not shown sinceitlieswell below theimage. � is thetop of theheadand is thebaseof thefeetof thepersonwhile � is theintersectionwith thevanishingline.

A map in the world betweenparallel planesinducesamap in the image betweenimagesof points on the twoplanes.This imagemapis aplanarhomology[15], whichisa planeprojective transformationwith five degreesof free-dom, having a line of fixed points, called the axis and adistinctfixedpointnotontheaxisknown asthevertex. Pla-narhomologiesarisenaturallyin animagewhentwo planesrelatedby a perspectivity in 3-spaceareimaged[16]. Thegeometryis illustratedin figure4.

In ourcasethevanishingline of theplane,andtheverti-cal vanishingpoint,are,respectively, theaxisandvertex ofthe homologywhich relatesa pair of planesin the pencil.This line andpoint specifyfour of thefive degreesof free-domof thehomology. Theremainingdegreeof freedomofthehomologyis uniquelydeterminedfrom any pair of im-agepointswhich correspondbetweentheplanes(points and � in figure4).

This meansthat we can comparemeasurementsmadeon two separateplanesby mappingbetweenthe planesinthe referencedirectionvia thehomology. In particularwemaycompute(i) theratiobetweentwo parallellengths,onelengthon eachplane;(ii) the ratio betweentwo areas,oneareaoneachplane.In factwecansimply transferall pointsfrom one plane to the referenceplane using the homol-

πX

T

B

π /

X / l

v

b

t

πx

π//

i

x

Figure 4: Homology mapping betweenparallel planes: (left) A point�on plane� is mappedinto thepoint � on � by a parallelprojection.

(right) In the imagethemappingbetweentheimagesof thetwo planesisa homology, with � thevertex and � theaxis. Thecorrespondence����fixestheremainingdegreeof freedomof thehomologyfromthecross-ratioof thefour points: � , � , � and .

Figure5: Measuring the ratio of lengthsof parallel line segmentslyingon two parallel sceneplanes:Thepoints � and (togetherwith theplanevanishingline andthe vanishingpoint) definethe homologybetweenthetwo planeson thefacadeof thebuilding.

ogy andthen,sincethe referenceplane’s vanishingline isknown,makeaffinemeasurementsin theplane,e.g.parallellengthor arearatios.Example. Figure5 shows thatgiventhe referencevanish-ing line andvanishingpoint,andapointcorrespondence(inthereferencedirection)oneachof two parallelplanes,thenthe ratio of lengthsof parallelline segmentsmaybe com-putedfrom the image. The formula usedto computethisresultis givenin section3.2.

2.3. Determining the cameraposition

In section2.1,wecomputeddistancesbetweenplanesasa ratio relative to thecamera’s distancefrom the referenceplane.Conversely, we maycomputethecamera’s distancefrom aparticularplaneknowing asinglereferencedistance.

Furthermore,by consideringfigure 1 it is seenthat thelocationof thecamerarelative to thereferenceplaneis theback-projectionof the vanishingpoint onto the referenceplane. This back-projectionis accomplishedby a homog-raphy which mapsthe imageto the referenceplane(andvice-versa).Althoughthechoiceof coordinateframein theworld is somewhatarbitrary, fixing this frameimmediatelydefinesthehomographyuniquelyandhencethecamerapo-sition.

We show anexamplein figure12,wherethelocationofthecameracentrehasbeendetermined,andsuperimposedinto avirtual view of thescene.

3. Algebraic Representation

Themeasurementsdescribedin theprevioussectionarecomputedin termsof cross-ratios.In this sectionwe de-velop a uniform algebraicapproachto the problemwhichhasanumberof advantagesoverdirectgeometricconstruc-tion: first, it avoidspotentialproblemswith orderingfor thecross-ratio;second,it enablesus to deal with both mini-malor over-constrainedconfigurationsuniformly; third, weunify the different typesof measurementwithin one rep-resentation;andfourth, in section4 we usethis algebraicrepresentationto developan uncertaintyanalysisfor mea-surements.

To begin wedefineanaffinecoordinatesystem� �"! inspace.Let theoriginof thecoordinateframelie ontherefer-enceplane,with the � and � -axesspanningtheplane.The! -axis is the referencedirection,which is thusany direc-tion not parallelto theplane.Theimagecoordinatesystemis theusual#%$ affineimageframe,andapoint

�in spaceis

projectedto theimagepoint & via a ')(+* projectionmatrix,as: &.- ,�� -0/2143517681:9;1=<?> �

where & and�

are homogeneousvectors in the form:& - @A#CBD$EBDFHGJI ,� -K@A�.BD�LBM!�B�NOGDI , and ‘ - ’ means

equalityup to scale.If we denotethe vanishingpoints for the � , � and !

directionsas(respectively) �CP , �:Q and� , thenit is clearbyinspectionthatthefirst threecolumnsof

,arethevanishing

points; �:PR- 143 , �:QS- 1:6 and �T- 179 , and that thefinal columnof

,is theprojectionof theorigin of theworld

coordinatesystem,UV- 1:< . Sinceour choiceof coordinateframehasthe � and� axesin thereferenceplane1W3 -X�=Pand 176 -��:Q aretwo distinctpointson thevanishingline.Choosingthesepointsfixesthe � and � affine coordinateaxes. We denotethevanishingline by � , andto emphasisethat the vanishingpoints � P and � Q lie on it, we denotethemby �ZY3 , ��Y6 , with �AY[]\ �C-_^ .

Columns1, 2 and4 of theprojectionmatrixarethethreecolumnsof thereferenceplaneto imagehomography. Thishomographymusthave rank three,otherwisethereference

planeto imagemapis degenerate.Consequently, thefinalcolumn(the origin of the coordinatesystem)mustnot lieon thevanishingline, sinceif it doesthenall threecolumnsarepointson the vanishingline, andthusarenot linearlyindependent.Hencewesetit to be U`- 1 < -X��acbdb �Jbebf-hg� .

Thereforethefinal parametrizationof theprojectionma-trix,

is: , - / � Y 3 � Y6 i � g � > (1)

where i is a scalefactor, which hasan importantrole toplay in theremainderof thepaper.

In the following sectionswe show how to computevariousmeasurementsfrom this projectionmatrix. Mea-surementsbetweenplanesareindependentof the first two(under-determined)columnsof

,. For thesemeasurements

theonly unknown quantityis i . Coordinatemeasurementswithin the planesdependon the first two and the fourthcolumnsof

,. They defineanaffinecoordinateframewithin

the plane. Affine measurements(e.g.arearatios),though,areindependentof theactualcoordinateframeanddependonly on the fourth columnof

,. If any metric information

on the planeis known, we may imposeconstraintson thechoiceof theframe.

3.1. Measurementsbetweenparallel planes

We wish to measurethe distancebetweensceneplanesspecifiedby a basepoint j on thereferenceplaneandtoppoint k in thescene.Thesepointsmaybechosenasrespec-tively @l�mBD�LBD^nG and @l�mBD�LBM!HG , andtheir imagesare and� . If

,is theprojectionmatrix thenthe imagecoordinates

are o- , pqqr � � ^ s tvuuw B �H- , pqqr � � ! s tvuuwTheequationsabovecanberewrittenas - xE@A� 1 3Wy � 1 6zy 1 < G (2)�{- |W@l� 1 3Wy � 1 6zy ! 1 9Ly 1 < G (3)

where x and | areunknown scalefactors,and 1 [ is the } thcolumnof the

,matrix.

Taking thescalarproductof (2) with g � yields x~-��� \ ,andcombiningthis with thethird columnof (1) and(3) weobtain i !_- � beb �(���beb@ � � \ 4G?beb ��(`��bdb (4)

Since i ! scaleslinearly with i we have obtainedaffinestructure.If i is known, thenweimmediatelyobtainamet-ric valuefor ! . Conversely, if ! is known (i.e. it is a refer-encedistance)thenwe have a meansof computingi , andhenceremoving theaffineambiguity.

Figure 6: Measuring heights using parallel lines: Given the verticalvanishingpoint, the vanishingline for the groundplaneanda referenceheight, the distanceof the top of the window on the right wall from thegroundplaneis measuredfrom the distancebetweenthe two horizontallinesshown, onedefinedby thetop edgeof thewindow, andtheotheronthegroundplane.

Example. In figure 6 heightsfrom the groundplanearemeasuredbetweentwo parallellines,oneoff theplane(top)andone on the plane(base). In fact, thanksto the planevanishingline,givenoneline parallelto thereferenceplaneit is easyto computethefamilyof parallellines.Computingthedistancebetweenthemis a straightforwardapplicationof (4).

3.2. Measurementson parallel planes

The projectionmatrix,

from the world to the imageisdefinedabovewith respectto acoordinateframeontheref-erenceplane. In this sectionwe determinethe projectionmatrix

, � referredto theparallelplane��� andweshow howthe homologybetweenthe two planescan be derived di-rectly from thetwo projectionmatrices.

Supposetheworld coordinatesystemis translatedfromtheplane� ontotheplane��� alongthereferencedirection,thenit is easyto show thatwecanparametrizethenew pro-jectionmatrix

, � as:, � - / 1W351:6 i � i !�� y g � >where ! is the distancebetweenthe planes. Note that if!X-_^ then

, �E- , correctly.Theplaneto imagehomographiescanbeextractedfrom

theprojectionmatricesignoringthethird column,to give:� - / 1435146 g � > B � � - / 1W35176 i !�� y g � >Then �� - � � �c� 3 mapsimagepointson the plane � ontopointson theplane��� andsodefinesthehomology.

A shortcomputationgivesthehomologymatrix �� as:�� -�� y i !�� � � I (5)

Given the homologybetweentwo planesin the pencil wecantransferall pointsfrom oneplaneto theotherandmakeaffinemeasurementsin theplane(seefig 5 andfig 7).

Figure 7: Measuring ratios of areason separateplanes: The imagepoints � and togetherwith thevanishingline of the two parallelplanesandthe vanishingpoint for the orthogonaldirectiondefinethehomologybetweentheplanes.Theratio betweentheareaof thewindow on theleftplaneandthatof thewindow on theright planeis computed.

3.3. Determining cameraposition

Supposethecameracentreis ��-�@l�+��BD���?B�!W�?B�No�MGJI inaffine coordinates(seefigure 1). Thensince

, �S-S� wehave , ��-_� Y3 ��� y � Y6 ��� y i �4!W� y g �ZNo�L-�� (6)

Thesolutionto thissetof equationsis given(usingCramer’srule)by�+��- � det / �ZY6 � g � > , ���o- det / �ZY3 � g � >i ! � - � det / �ZY3 �AY6 g � > , N � - det / �AY3 �ZY6 � > (7)

Notethatonceagainweobtainstructureoff theplaneup tothe affine scalefactor i . As before,we may upgradethedistanceto metricwith knowledgeof i , or useknowledgeof cameraheightto computei andupgradetheaffinestruc-ture.

Note that affine viewing conditions(wherethe cameracentreis at infinity) presentno problemto theexpressionsin (7), since in this casewe have g �m- / ^�^ � > I and��- / ����^ > I . Hence N � -�^ so we obtaina cam-eracentreon theplaneat infinity, aswewouldexpect.Thispointon ��� representstheviewing directionfor theparal-lel projection.

If the viewpoint is finite (i.e. not affine viewing condi-tions) thenthe formula for i !z� may be developedfurtherby taking the scalarproductof both sidesof (6) with thevanishingline g � . Theresultis: i !z�L- � @�g � \ �7G � 3 .4. Uncertainty Analysis

Featuredetectionandextraction– whethermanualor au-tomatic(e.g.usinganedgedetector)– canonly beachieved

bΛ

tΛ

b

t t

bFigure 8: Maximum likelihood estimation of the top and basepoints(closeupof fig. 9): (left) The top andbaseuncertaintyellipses,respec-tively �%  and ��¡ , areshown. Theseellipsesarespecifiedby theuser, andindicateaconfidenceregion for localizingthepoints.(right) MLE topandbasepoints ¢� and ¢ arealignedwith theverticalvanishingpoint (outsidetheimage).

to a finite accuracy. Any featuresextractedfrom animage,therefore,aresubjectto errors. In this sectionwe considerhow theseerrorspropagatethroughthe measurementfor-mulaein orderto quantifytheuncertaintyon thefinal mea-surements.

When making measurementsbetweenplanes, uncer-tainty arisesfrom theuncertaintyin

,, andfrom theuncer-

tain imagelocationsof thetopandbasepoints � and . Theuncertaintyin

,dependson the locationof the vanishing

line, thelocationof thevanishingpoint,andon i , theaffinescalefactor. Sinceonly the final two columnscontribute,we modeltheuncertaintyin

,asa £�(~£ homogeneousco-

variancematrix, ¤z¥ . Sincethetwo columnshave only fivedegreesof freedom(two for � , two for � andone for i ),the covariancematrix is singular, with rank five. Detailsof its computationaregivenin [4] andareomittedhereforbrevity.

Likewise,theuncertaintyin thetop andbasepoints(re-sulting largely from the finite accuracy with which thesefeaturesmaybelocatedin theimage)is modelledby covari-ancematrices¤L¦ and ¤z§ . Sincein theerror-freecase,thesepointsmustbealignedwith theverticalvanishingpoint wecandeterminemaximumlikelihoodestimatesof their truelocations( �� and � ) by minimisingtheobjective@l 6 � g 6 G I4¨ � 3¦�© @A 6 � g 6 G y @l� 6 � g� 6 G I4¨ � 3§J© @A� 6 � g� 6 G(which is the sum of the Mahalanobisdistancesbetweenthe input pointsandthe ML estimates,the subscript2 in-dicatesinhomogeneous2-vectors)subjectto thealignmentconstraint � \ @ g�"(_g7G�-O^ . Usingstandardtechniques[7]weobtainafirst orderapproximationto the *ª(�* , rankthreecovarianceof theparametersg« -¬@ �� I6 � I6 GJI . Figure8illustratestheidea.

Figure 9: Uncertainty analysison height measurements: The imageshown wascapturedfrom a cheapsecuritytype camerawhich exhibitedradial distortion. This hasbeencorrectedandthe heightof the manes-timated(measurementsare in cm). (left) The heightof the manandtheassociateduncertaintyarecomputedas190.6cm(c.f. groundtruth value190cm). The vanishingline for the groundplaneis shown in white atthe top of the image. Whenonereferenceheight is usedthe uncertainty(3-sigma)is ­:®°¯²± cm,while (right) it reducesto ­7³´¯ µ cm astwo moreref-erenceheightsareintroduced(thefiling cabinetandthetableon theleft).

Now, assumingthe statisticalindependenceof g« and,

weobtainafirst orderapproximationfor thevarianceof thedistancemeasurement:¶ 6· -�¸ ·+¹ ¨`º» ¼¼ ¨ ¥ ½ ¸ · I (8)

where ¸ · is thes ( s ^ Jacobianmatrix of thefunction

whichmapstheprojectionmatrixandtopandbasepointstoadistancebetweenthem(4). Thevalidity of all approxima-tion hasbeentestedby Monte Carlo simulationsandby anumberof measurementson real imageswherethegroundtruthwasknown.Example. An imageobtainedfrom a poorquality securitycamerais shown in figure 9. It hasbeencorrectedfor ra-dial distortionusing the methoddescribedin [6], and thefloor taken asthe referenceplane. Vertical andhorizontallines areusedto computethe

,matrix of the scene.One

referenceheight is usedto obtainthe affine scalefactor ifrom (4), so othermeasurementsin the samedirectionaremetric.

The computedheight of the man and an associated3-standarddeviation uncertaintyaredisplayedin the figure.The heightobtaineddiffersby only 6mm from the knowntrue value. As the numberof referencedistancesis in-creased,so the uncertaintyon

,(in fact just on i ) de-

creases,resultingin a decreasein uncertaintyof the mea-suredheight,astheoreticallyexpected.

5. Applications

5.1. Forensicscience

A commonrequirementin surveillanceimagesis to ob-tain measurementsfrom the scene,suchastheheightof afelon. Although, the felon hasusuallydepartedthescene,referencelengthscanbemeasuredfrom fixturessuchasta-blesandwindows.

Figure 10: Measuring the height of a personin an outdoor scene:Thegroundplaneis the referenceplane,and its vanishingline is computedfrom theslabson thefloor. Theverticalvanishingpoint is computedfromtheedgesof thephonebox,whoseheightis known andusedasreference.Theveridicalheightis 187cm,but notethat thepersonis leaningslightlyonhis right foot.

Figure11: Measuring heightsof objectson separateplanes:Usingthehomologybetweenthegroundplane(initial reference)andtheplaneof thetable,wecandeterminetheheightof thefile on thetable.

In figure10 the edgesof the paving stoneson the floorareusedto computethevanishingline of thegroundplane;theedgesof thephoneboxto computetheverticalvanishingpoint; andtheheightof thephonebox providesthemetriccalibrationin theverticaldirection.Theheightof thepersonis thencomputedusing(4).

Figure11showsanexamplewherethehomologyis usedto projectpointsbetweenplanesso thata verticaldistancemay be measuredgiven the distancebetweena planeandthereferenceplane.

5.2. Virtual modelling

In figure 12 we show an exampleof complete3D re-constructionof a scene.Two setsof horizontaledgesareusedto computethe vanishingline for the groundplane,andvertical edgesusedto computethe vertical vanishingpoint. Fourpointswith known Euclideancoordinatesdeter-minethemetriccalibrationof thegroundplaneandthusforthe pencil of horizontalplaneswhich sharethe vanishingline. Thedistanceof the top of thewindow to theground,and the height of one of the pillars areusedas reference

Figure 12: Complete3D reconstructionof a real scene:(left) originalimage;(right) a view of thereconstructed3D model;(bottom)A view ofthereconstructed3D modelwhichshows thepositionof thecameracentre(planelocationX,Y andheight)with respectto thescene.

lengths.Thepositionof thecameracentreis alsoestimatedandis shown in thefigure.

5.3. Modelling paintings

Figure 13 shows a masterpieceof Italian Renaissancepainting, “La Flagellazionedi Cristo” by Piero dellaFrancesca(1416- 1492).Thepaintingfaithfully followsthegeometricrulesof perspective,andthereforewe canapplythe methodsdevelopedhereto obtaina correct3D recon-structionof thescene.

Unlike other techniques[8] whosemain aim is to cre-ateconvincing new views of thepaintingregardlessof thecorrectnessof the3D geometry, herewe reconstructa geo-metricallycorrect3D modelof theviewedscene.

In thepaintinganalyzedhere,thegroundplaneis chosenasreferenceand its vanishingline canbe computedfromtheseveralparallellineson it. Theverticalvanishingpointfollowsfrom theverticallinesandconsequentlytherelativeheightsof peopleandcolumnscanbecomputed.Further-morethegroundplanecanberectifiedfrom thesquarefloorpatternsand thereforethe positionon the groundof each

verticalobjectestimated[5, 10]. Themeasurements,up toanoverall scalefactor, areusedto computea threedimen-sionalVRML modelof the scene.Two differentviews ofthemodelareshown in figure13.

6. Summary and Conclusions

We have explored how the affine structureof 3-spacemay be partially recovered from perspective images intermsof a setof planesparallelto a referenceplaneandareferencedirectionnotparallelto thereferenceplane.Moregenerally, affine3 spacemayberepresentedentirelyby setsof parallelplanesanddirections[2]. We arecurrently in-vestigatinghow this full geometryis bestrepresentedandcomputedfrom a singleperspectiveimage.

References

[1] L. B. Alberti. DePictura. Laterza,1980.[2] M. Berger. GeometryII . Springer-Verlag,1987.[3] R. T. Collins andR. S. Weiss. Vanishingpoint calculation

asa statisticalinferenceon theunit sphere.In Proc. ICCV,pages400–403,Dec1990.

[4] A. Criminisi, I. Reid,andA. Zisserman.Computing3D eu-clideandistancefromasingleview. TechnicalReportOUEL2158/98,Dept.Eng.Science,Universityof Oxford,1998.

[5] A. Criminisi, I. Reid,andA. Zisserman.A planemeasuringdevice. ImageandVisionComputing, 17(8):625–634,1999.

[6] F. DevernayandO. Faugeras.Automaticcalibrationandre-moval of distortionfrom scenesof structuredenvironments.In SPIE, volume2567,SanDiego,CA, Jul1995.

[7] O. Faugeras.Three-DimensionalComputerVision: a Geo-metricViewpoint. MIT Press,1993.

[8] Y. Horry, K. Anjyo, andK. Arai. Tourinto thepicture:Usinga spiderymeshinterfaceto make animationfrom a singleimage.In Proc.ACM SIGGRAPH, pages225–232,1997.

[9] T. Kim, Y. Seo,andK. Hong. Physics-based3D positionanalysisof a soccerball from monocularimagesequences.Proc. ICCV, pages721– 726,1998.

[10] D. Liebowitz, A. Criminisi, andA. Zisserman.Creatingar-chitecturalmodelsfromimages.In Proc.EuroGraphics, Sep1999.

[11] D. Liebowitz andA. Zisserman.Metric rectificationfor per-spective imagesof planes.In Proc.CVPR, pages482–488,Jun1998.

[12] G. F. McLeanandD. Kotturi. Vanishingpoint detectionbyline clustering.IEEET-PAMI, 17(11):1090–1095,1995.

[13] M. Proesmans,T. Tuytelaars,and L. Van Gool. Monoc-ular image measurements. Technical Report Improofs-M12T21/1/P, K.U.Leuven,1998.

[14] I. ReidandA. Zisserman.Goal-directedvideometrology. InR. Cipolla andB. Buxton,editors,Proc. ECCV, volumeII,pages647–658.Springer, Apr 1996.

[15] C.E.Springer. GeometryandAnalysisof ProjectiveSpaces.Freeman,1964.

[16] L. VanGool, M. Proesmans,andA. Zisserman.Planarho-mologiesasabasisfor groupingandrecognition.ImageandVision Computing, 16:21–26,Jan1998.

Figure 13: Complete 3D reconstruction of a Renaissancepainting:(top) La Flagellazionedi Cristo, (1460,Urbino, GalleriaNazionaledelleMarche). (middle)A view of thereconstructed3D model. Thepatternedfloor hasbeenreconstructedin areaswhereit is occludedby takingadvan-tageof the symmetryof its pattern. (bottom)anotherview of the modelwith theroof removed to show therelative positionsof peopleandarchi-tecturalelementsin thescene.Notetherepeatedgeometricpatternon thefloor in theareadelimitedby thecolumns(barelyvisible in thepainting).Notethatthepeoplearerepresentedsimplyasflat silhouettessinceit is notpossibleto recover their volumefrom oneimage,they have beencut outmanuallyfrom theoriginal image.Thecolumnshave beenapproximatedwith cylinders.