Towards aTowards aMultiscale Figural Geometry Multiscale Figural Geometry
Towards aTowards aMultiscale Figural Geometry Multiscale Figural Geometry
Stephen PizerStephen PizerAndrew Thall, Paul YushkevichAndrew Thall, Paul Yushkevich
www.cs.unc.edu/Research/Imagewww.cs.unc.edu/Research/Image
Medical Image Display & Analysis GroupMedical Image Display & Analysis GroupUniversity of North Carolina, Chapel HillUniversity of North Carolina, Chapel Hill
Acknowledgements: James Chen, Guido Gerig, and P. Thomas Fletcher for figures, Acknowledgements: James Chen, Guido Gerig, and P. Thomas Fletcher for figures, NIH grant P01 CA47982, NSF grant CCR-9910419, and Intel for a computer grantNIH grant P01 CA47982, NSF grant CCR-9910419, and Intel for a computer grant
Stephen PizerStephen PizerAndrew Thall, Paul YushkevichAndrew Thall, Paul Yushkevich
www.cs.unc.edu/Research/Imagewww.cs.unc.edu/Research/Image
Medical Image Display & Analysis GroupMedical Image Display & Analysis GroupUniversity of North Carolina, Chapel HillUniversity of North Carolina, Chapel Hill
Acknowledgements: James Chen, Guido Gerig, and P. Thomas Fletcher for figures, Acknowledgements: James Chen, Guido Gerig, and P. Thomas Fletcher for figures, NIH grant P01 CA47982, NSF grant CCR-9910419, and Intel for a computer grantNIH grant P01 CA47982, NSF grant CCR-9910419, and Intel for a computer grant
Intrinsic Object-Based Geometry Intrinsic Object-Based Geometry Suitable for Shape DescriptionSuitable for Shape Description
Intrinsic Object-Based Geometry Intrinsic Object-Based Geometry Suitable for Shape DescriptionSuitable for Shape Description
The needThe need: object-based positional, : object-based positional, orientational, and metric correspondence orientational, and metric correspondence among topologically figurally equivalent among topologically figurally equivalent objects or groups of objectsobjects or groups of objects Boundary of objectBoundary of object In interior of objectIn interior of object Exterior to object, Exterior to object,
between objectsbetween objects
Suitability for shape description impliesSuitability for shape description implies Magnification invarianceMagnification invariance At all levels of spatial scale (locality)At all levels of spatial scale (locality)
The needThe need: object-based positional, : object-based positional, orientational, and metric correspondence orientational, and metric correspondence among topologically figurally equivalent among topologically figurally equivalent objects or groups of objectsobjects or groups of objects Boundary of objectBoundary of object In interior of objectIn interior of object Exterior to object, Exterior to object,
between objectsbetween objects
Suitability for shape description impliesSuitability for shape description implies Magnification invarianceMagnification invariance At all levels of spatial scale (locality)At all levels of spatial scale (locality)
Definition of Spatial ScaleDefinition of Spatial ScaleDefinition of Spatial ScaleDefinition of Spatial Scale
Scale: Scale: There are two separate and different notions:There are two separate and different notions: Spatial coverage of each geometric elementSpatial coverage of each geometric element Distance of inter-element communicationDistance of inter-element communication
Scale: Scale: There are two separate and different notions:There are two separate and different notions: Spatial coverage of each geometric elementSpatial coverage of each geometric element Distance of inter-element communicationDistance of inter-element communication
Mesh of voxels Boundary atom mesh Mesh of voxels Boundary atom mesh Medial atom meshMedial atom mesh
Multiple Spatial ScalesMultiple Spatial ScalesMultiple Spatial ScalesMultiple Spatial Scales
Scale aspects Scale aspects Geometric element coverageGeometric element coverage Inter-element communication distanceInter-element communication distance
Thesis: The two measures need to be similarThesis: The two measures need to be similar Multiple scale levelsMultiple scale levels
Scale aspects Scale aspects Geometric element coverageGeometric element coverage Inter-element communication distanceInter-element communication distance
Thesis: The two measures need to be similarThesis: The two measures need to be similar Multiple scale levelsMultiple scale levels
Mesh of voxels Mesh of voxels Medial atom meshMedial atom mesh
Figural Geometry (position, orientation, Figural Geometry (position, orientation, local size) Comes from Medial Atomslocal size) Comes from Medial Atoms
Figural Geometry (position, orientation, Figural Geometry (position, orientation, local size) Comes from Medial Atomslocal size) Comes from Medial Atoms
Medial atomsMedial atoms (1st order medial locus)(1st order medial locus) xx, , FF = (= (bb,,n,bn,b) frame, r, ) frame, r, (object angle) (object angle)
bb in direction of minimum dr/ds (- in direction of minimum dr/ds (-xxr)r)bb in level direction of r [3D] in level direction of r [3D]nn is normal to medial skeleton is normal to medial skeleton
Medial atomsMedial atoms (1st order medial locus)(1st order medial locus) xx, , FF = (= (bb,,n,bn,b) frame, r, ) frame, r, (object angle) (object angle)
bb in direction of minimum dr/ds (- in direction of minimum dr/ds (-xxr)r)bb in level direction of r [3D] in level direction of r [3D]nn is normal to medial skeleton is normal to medial skeleton
Figurally RelevantFigurally RelevantSpatial Scale LevelsSpatial Scale LevelsFigurally RelevantFigurally Relevant
Spatial Scale LevelsSpatial Scale Levels Multiple objectsMultiple objects Individual objectIndividual object
i.e., multiple figuresi.e., multiple figures Individual figureIndividual figure
mesh of medial atomsmesh of medial atoms Figural sectionFigural section
i.e., multiple figural sectionsi.e., multiple figural sections figural section centered at medial atomfigural section centered at medial atom
Figural section more finely spaced, ..Figural section more finely spaced, .. Boundary sectionBoundary section Boundary section more finely spaced, ...Boundary section more finely spaced, ...
Multiple objectsMultiple objects Individual objectIndividual object
i.e., multiple figuresi.e., multiple figures Individual figureIndividual figure
mesh of medial atomsmesh of medial atoms Figural sectionFigural section
i.e., multiple figural sectionsi.e., multiple figural sections figural section centered at medial atomfigural section centered at medial atom
Figural section more finely spaced, ..Figural section more finely spaced, .. Boundary sectionBoundary section Boundary section more finely spaced, ...Boundary section more finely spaced, ...
medial atom
Figural Types and the Manifold Figural Types and the Manifold of Medial Atomsof Medial Atoms
Figural Types and the Manifold Figural Types and the Manifold of Medial Atomsof Medial Atoms
Slab
Tube
M-rep Boundary implied from interpolated continuous manifold of medial atoms
Magnification Invariance at Magnification Invariance at All Spatial Scale LevelsAll Spatial Scale Levels
Magnification Invariance at Magnification Invariance at All Spatial Scale LevelsAll Spatial Scale Levels
Inside boundary featuresInside boundary features radius of curvature-radius of curvature-
proportional distancesproportional distances Inside figural sectionsInside figural sections
r-proportional distancesr-proportional distances Inside individual figuresInside individual figures
r-proportional distancesr-proportional distances
Inside boundary featuresInside boundary features radius of curvature-radius of curvature-
proportional distancesproportional distances Inside figural sectionsInside figural sections
r-proportional distancesr-proportional distances Inside individual figuresInside individual figures
r-proportional distancesr-proportional distances
Magnification Invariance at Magnification Invariance at All Spatial Scale LevelsAll Spatial Scale Levels
Magnification Invariance at Magnification Invariance at All Spatial Scale LevelsAll Spatial Scale Levels
Individual objectIndividual object In interface between figuresIn interface between figures
blended r-proportional distances blended r-proportional distances Multiple objectsMultiple objects
Outside objectsOutside objects blended r-proportional distancesblended r-proportional distances concavities’ effect disappear with concavities’ effect disappear with
distancedistance
Individual objectIndividual object In interface between figuresIn interface between figures
blended r-proportional distances blended r-proportional distances Multiple objectsMultiple objects
Outside objectsOutside objects blended r-proportional distancesblended r-proportional distances concavities’ effect disappear with concavities’ effect disappear with
distancedistance
Figural (Medially based) Geometry Figural (Medially based) Geometry Figural (Medially based) Geometry Figural (Medially based) Geometry
Locally magnification invariant Locally magnification invariant means r-proportional distancesmeans r-proportional distances Along medial skeletonAlong medial skeleton Along medial sails (implied boundary Along medial sails (implied boundary
normals)normals)
Medially (figurally) based coordinate Medially (figurally) based coordinate system provides intrinsic coordinatessystem provides intrinsic coordinates Along medial skeletonAlong medial skeleton Along medial sails (implied boundary Along medial sails (implied boundary
normals)normals) Overall metric??Overall metric??
Locally magnification invariant Locally magnification invariant means r-proportional distancesmeans r-proportional distances Along medial skeletonAlong medial skeleton Along medial sails (implied boundary Along medial sails (implied boundary
normals)normals)
Medially (figurally) based coordinate Medially (figurally) based coordinate system provides intrinsic coordinatessystem provides intrinsic coordinates Along medial skeletonAlong medial skeleton Along medial sails (implied boundary Along medial sails (implied boundary
normals)normals) Overall metric??Overall metric??
Spatial coordinates capable of Spatial coordinates capable of providing correspondence at any scaleproviding correspondence at any scale
Spatial coordinates capable of Spatial coordinates capable of providing correspondence at any scaleproviding correspondence at any scale
Medial coordinates (u[,v])Medial coordinates (u[,v])continuous, integer multiples of continuous, integer multiples of r r
at samples, where at samples, where is scale level is scale level r-proportional along medial surfacer-proportional along medial surface
Boundary coordinates (u[,v],t)Boundary coordinates (u[,v],t) Spatial coordinates (u[,v],t,d/r)Spatial coordinates (u[,v],t,d/r)
From implied boundary along From implied boundary along geodesic of distance that at geodesic of distance that at boundary is in normal directionboundary is in normal direction
Medial coordinates (u[,v])Medial coordinates (u[,v])continuous, integer multiples of continuous, integer multiples of r r
at samples, where at samples, where is scale level is scale level r-proportional along medial surfacer-proportional along medial surface
Boundary coordinates (u[,v],t)Boundary coordinates (u[,v],t) Spatial coordinates (u[,v],t,d/r)Spatial coordinates (u[,v],t,d/r)
From implied boundary along From implied boundary along geodesic of distance that at geodesic of distance that at boundary is in normal directionboundary is in normal direction
Inside object: (u[,v],t,d/r)Inside object: (u[,v],t,d/r) (u,v) give multiples of r(u,v) give multiples of r distance on medial sheet along geodesics of r-distance on medial sheet along geodesics of r-
proportional distanceproportional distance Outside objectOutside object
Near boundary (inside focal surface): Near boundary (inside focal surface): (u[,v],t,d/r)(u[,v],t,d/r)
Far outside boundary: (u[,v],t,d/r) via distance Far outside boundary: (u[,v],t,d/r) via distance (scale) related figural convexification(scale) related figural convexification
geodesics do not crossgeodesics do not cross
Inside object: (u[,v],t,d/r)Inside object: (u[,v],t,d/r) (u,v) give multiples of r(u,v) give multiples of r distance on medial sheet along geodesics of r-distance on medial sheet along geodesics of r-
proportional distanceproportional distance Outside objectOutside object
Near boundary (inside focal surface): Near boundary (inside focal surface): (u[,v],t,d/r)(u[,v],t,d/r)
Far outside boundary: (u[,v],t,d/r) via distance Far outside boundary: (u[,v],t,d/r) via distance (scale) related figural convexification(scale) related figural convexification
geodesics do not crossgeodesics do not cross
Figural Coordinates for Single FigureFigural Coordinates for Single Figure Figural Coordinates for Single FigureFigural Coordinates for Single Figure
Inside figures not near hingesInside figures not near hinges same as for single figuresame as for single figure
Outside object: see two slides laterOutside object: see two slides later
Inside figures not near hingesInside figures not near hinges same as for single figuresame as for single figure
Outside object: see two slides laterOutside object: see two slides later
Figural Coordinates for Object Figural Coordinates for Object Made From Multiple Attached FiguresMade From Multiple Attached Figures
Figural Coordinates for Object Figural Coordinates for Object Made From Multiple Attached FiguresMade From Multiple Attached Figures
Blend in hinge regionsBlend in hinge regions w=(dw=(d11/r/r11 - d - d2 2 /r/r22 )/T )/T Blended d/r when |w| <1 and u-uBlended d/r when |w| <1 and u-u00 < T < T Implicit boundary: (u,w, t)Implicit boundary: (u,w, t) Implicit normals and geodesicsImplicit normals and geodesics
Blend in hinge regionsBlend in hinge regions w=(dw=(d11/r/r11 - d - d2 2 /r/r22 )/T )/T Blended d/r when |w| <1 and u-uBlended d/r when |w| <1 and u-u00 < T < T Implicit boundary: (u,w, t)Implicit boundary: (u,w, t) Implicit normals and geodesicsImplicit normals and geodesics
Figural Coordinates for Object Figural Coordinates for Object Made From Multiple Attached FiguresMade From Multiple Attached Figures
Figural Coordinates for Object Figural Coordinates for Object Made From Multiple Attached FiguresMade From Multiple Attached Figures
Near boundary: via blendingNear boundary: via blending Far outside boundaryFar outside boundary
same convexification principle as with same convexification principle as with single figuressingle figures
blend geodesics according to dblend geodesics according to dkk/r/rkk
Near boundary: via blendingNear boundary: via blending Far outside boundaryFar outside boundary
same convexification principle as with same convexification principle as with single figuressingle figures
blend geodesics according to dblend geodesics according to dkk/r/rkk
Figural Coordinates between Objects Figural Coordinates between Objects Figural Coordinates between Objects Figural Coordinates between Objects
Uses of CorrespondenceUses of CorrespondenceUses of CorrespondenceUses of Correspondence
Geometric typicality (segment’n prior)Geometric typicality (segment’n prior) by boundary point to boundary point correspondenceby boundary point to boundary point correspondence
Geometric representation to image match measureGeometric representation to image match measure by boundary-relative correspondenceby boundary-relative correspondence
in collar about boundary out to fixed distance via metricin collar about boundary out to fixed distance via metric union of collar and interior of objectunion of collar and interior of object
For homologies used in statistical shape For homologies used in statistical shape characterization: leads to localitycharacterization: leads to locality
For elements in mechanical calculationsFor elements in mechanical calculations For comparison of segmented For comparison of segmented
object to true object object to true object
Geometric typicality (segment’n prior)Geometric typicality (segment’n prior) by boundary point to boundary point correspondenceby boundary point to boundary point correspondence
Geometric representation to image match measureGeometric representation to image match measure by boundary-relative correspondenceby boundary-relative correspondence
in collar about boundary out to fixed distance via metricin collar about boundary out to fixed distance via metric union of collar and interior of objectunion of collar and interior of object
For homologies used in statistical shape For homologies used in statistical shape characterization: leads to localitycharacterization: leads to locality
For elements in mechanical calculationsFor elements in mechanical calculations For comparison of segmented For comparison of segmented
object to true object object to true object
Open Geometric QuestionsOpen Geometric QuestionsOpen Geometric QuestionsOpen Geometric Questions
Full space metricFull space metric Outside figure convexificationOutside figure convexification Reflecting scale levelReflecting scale level
Representing toleranceRepresenting tolerance Controlling IIControlling IImedial locusmedial locus, D, Dxx
22r, r, xxrr Principled means forPrincipled means for
Inter-figural blending of figural metrics for attached Inter-figural blending of figural metrics for attached figuresfigures
Inter-object blending of object metricsInter-object blending of object metrics
Full space metricFull space metric Outside figure convexificationOutside figure convexification Reflecting scale levelReflecting scale level
Representing toleranceRepresenting tolerance Controlling IIControlling IImedial locusmedial locus, D, Dxx
22r, r, xxrr Principled means forPrincipled means for
Inter-figural blending of figural metrics for attached Inter-figural blending of figural metrics for attached figuresfigures
Inter-object blending of object metricsInter-object blending of object metrics
Figural (Medially based) GeometryFigural (Medially based) GeometryInternal points on single figureInternal points on single figure
Figural (Medially based) GeometryFigural (Medially based) GeometryInternal points on single figureInternal points on single figure
Sails are separate (Sails are separate (>0)>0) Both sails move with motion on medial surface Both sails move with motion on medial surface
Sails are separate (Sails are separate (>0)>0) Both sails move with motion on medial surface Both sails move with motion on medial surface
p
p
x
s
b
n
Figural (Medially based) GeometryFigural (Medially based) GeometryBranches and EndsBranches and Ends
Figural (Medially based) GeometryFigural (Medially based) GeometryBranches and EndsBranches and Ends
EndsEnds Sails come together (Sails come together (=0)=0) Boundary is vertex (2D) or crest (3D)Boundary is vertex (2D) or crest (3D) Medial disk or ball osculatesMedial disk or ball osculates
BranchesBranches Medial disk or ball tritangentMedial disk or ball tritangent Swallowtail of medial atomSwallowtail of medial atom
Retrograde motion of one sailRetrograde motion of one sail
EndsEnds Sails come together (Sails come together (=0)=0) Boundary is vertex (2D) or crest (3D)Boundary is vertex (2D) or crest (3D) Medial disk or ball osculatesMedial disk or ball osculates
BranchesBranches Medial disk or ball tritangentMedial disk or ball tritangent Swallowtail of medial atomSwallowtail of medial atom
Retrograde motion of one sailRetrograde motion of one sail
Geometrically Geometrically smaller scale smaller scale Interpolate (1st order) finer spacing of Interpolate (1st order) finer spacing of
atomsatoms Residual atom change, i.e., localResidual atom change, i.e., local
ProbabilityProbability At any scale, relates figurally homologous At any scale, relates figurally homologous
pointspoints Markov random field relating medial Markov random field relating medial
atom withatom with its immediate neighbors at that scaleits immediate neighbors at that scale its parent atom at the next larger scale and the its parent atom at the next larger scale and the
corresponding positioncorresponding position its children atoms its children atoms
Geometrically Geometrically smaller scale smaller scale Interpolate (1st order) finer spacing of Interpolate (1st order) finer spacing of
atomsatoms Residual atom change, i.e., localResidual atom change, i.e., local
ProbabilityProbability At any scale, relates figurally homologous At any scale, relates figurally homologous
pointspoints Markov random field relating medial Markov random field relating medial
atom withatom with its immediate neighbors at that scaleits immediate neighbors at that scale its parent atom at the next larger scale and the its parent atom at the next larger scale and the
corresponding positioncorresponding position its children atoms its children atoms
Multiscale Geometry and Probability Multiscale Geometry and Probability for a Figurefor a Figure
Multiscale Geometry and Probability Multiscale Geometry and Probability for a Figurefor a Figure
coarse, global
coarse resampled
fine, local