Recognizing Recognizing Deformable Deformable
ShapesShapesSalvador Ruiz CorreaSalvador Ruiz Correa
Ph.D. Thesis, Electrical Ph.D. Thesis, Electrical EngineeringEngineering
Basic IdeaBasic Idea
Generalize existing Generalize existing numeric surface numeric surface
representationsrepresentations for matching 3-D objects for matching 3-D objects to the problem of to the problem of identifying shape classes identifying shape classes allowing for shape deformationsallowing for shape deformations..
What Kind Of Deformations?What Kind Of Deformations?
Toy animals
3-D Faces
Normal
Mandibles
Neurocranium
Normal
Abnormal
Abnormal
Shape classes: significantamount of intra-class variability
DeformedDeformed Infants’ Skulls Infants’ Skulls
Sagittal
Coronal
NormalSagittal
Synostosis
BicoronalSynostosis
FusedSutures
Metopic
Occurs when sutures of the cranium fuse prematurely (synostosis).
Alignment-VerificationAlignment-Verification LimitationsLimitations
The approach does not extend well to the problemThe approach does not extend well to the problem
of identifying classes of similar shapes. In general:of identifying classes of similar shapes. In general:
Numeric shape representations are Numeric shape representations are not robust to not robust to deformations.deformations.
There are There are not exact correspondencesnot exact correspondences between between model and scene.model and scene.
Objects in a shape class Objects in a shape class do not aligndo not align..
AssumptionsAssumptions All shapes are represented as oriented surface All shapes are represented as oriented surface
meshes of fixed resolution.meshes of fixed resolution.
The The verticesvertices of the meshes in the of the meshes in the training settraining set are are in full correspondence.in full correspondence.
Finding full correspondences : hard problem yes … Finding full correspondences : hard problem yes … but it is approachable ( use but it is approachable ( use morphable models morphable models techniquetechnique:: Blantz and Vetter, SIGGRAPH 99; C. R. Blantz and Vetter, SIGGRAPH 99; C. R. Shelton, IJCV, 2000; Allen et al., SIGGRAPH 2003).Shelton, IJCV, 2000; Allen et al., SIGGRAPH 2003).
Four Key Elements To Our Four Key Elements To Our ApproachApproach
Architectureof
Classifiers
NumericSignatures
Components
SymbolicSignatures
4
+
1
2
3
Recognition And Recognition And Classification OfClassification Of
Deformable Deformable Shapes Shapes
The Spin Image SignatureThe Spin Image Signature
P
Xn
P is the selected vertex.
X is a contributing point of the mesh.
is the perpendicular distance from X to P’s surface normal.
is the signed perpendicular distance from X to P’s tangent plane.
tangent plane at P
Numeric Signatures: Spin Numeric Signatures: Spin ImagesImages
Rich set of surface shape descriptors.Rich set of surface shape descriptors.
Their spatial scale can be modified to include local and Their spatial scale can be modified to include local and non-local surface features. non-local surface features.
Representation is robust to scene clutter and occlusions.Representation is robust to scene clutter and occlusions.
P
Spin images for point P
3-D faces
Shape Class Components: Clusters Shape Class Components: Clusters of 3D Points with Similar Spin of 3D Points with Similar Spin
ImagesImages
……
……
ComponentComponentDetectorDetector
ComputeComputeNumericNumeric
SignaturesSignatures
Training SetTraining SetSelectSelectSeedSeedPointsPoints
RegionRegionGrowingGrowing
AlgorithmAlgorithm
Grown componentsGrown componentsaround seedsaround seeds
Labeled Labeled Surface MeshSurface Mesh
Selected 8 seedSelected 8 seedpoints by handpoints by hand
Component Extraction Component Extraction ExampleExample
Region Region GrowingGrowing
Grow one region at the time Grow one region at the time (get one detector(get one detectorper component)per component)
DetectedDetectedcomponents on acomponents on atraining sampletraining sample
How To Combine Component How To Combine Component Information?Information?
…Extracted components on test samples
12
3
76
4
8
5
1112 2 222 2
Note: Numeric signatures are invariant to mirror symmetry;our approach preserves such an invariance.
Symbolic SignatureSymbolic Signature
Symbolic Symbolic Signature at PSignature at P
3344556688 77
Labeled Labeled Surface MeshSurface Mesh
Matrix storing Matrix storing componentcomponent
labelslabels
EncodeEncodeGeometricGeometric
ConfigurationConfiguration
Critical Point P
Symbolic Signatures Are Symbolic Signatures Are Robust Robust
To DeformationsTo Deformations
PP3344
55
66 7788
3333 33 3344 44 44 44
88 88 88 8855 55 55 55
666666 77 77 77 7766
Relative position of Relative position of components is stable across components is stable across deformations: experimental deformations: experimental
evidenceevidence
Proposed ArchitectureProposed Architecture(Classification Example)(Classification Example)
InputInput
LabeledLabeledMeshMesh
ClasClasss
LabeLabell
-1-1(Abnormal(Abnormal
))
Verify spatial configuration
of the components IdentifyIdentify
SymbolicSymbolicSignaturesSignatures
IdentifyIdentifyComponentsComponents
Two classification Two classification stagesstages
Surface Surface MeshMesh
Architecture Architecture ImplementationImplementation
ALL our classifiers are (off-the-shelf) ALL our classifiers are (off-the-shelf) νν--Support Vector Machines (Support Vector Machines (νν--SVMsSVMs) ) (Sch(Schöölkopf et al., 2000 and 2001).lkopf et al., 2000 and 2001).
Component (and symbolic signature) Component (and symbolic signature) detectors are detectors are one-class classifiers.one-class classifiers.
Component label assignment: Component label assignment: performed with a performed with a multi-way classifiermulti-way classifier that uses that uses pairwise classification pairwise classification scheme.scheme.
Gaussian kernel. Gaussian kernel.
Experimental ValidationExperimental Validation
Recognition Tasks: 4 (T1 - T4)Recognition Tasks: 4 (T1 - T4)
Classification Tasks: 3 (T5 – T7)Classification Tasks: 3 (T5 – T7)
No. Experiments: 5470 No. Experiments: 5470
SetupSetup
Recognition Recognition Classification Classification
LaserLaserRotary TableRotary Table
Shape ClassesShape Classes
Enlarging Training Sets Using Enlarging Training Sets Using Virtual SamplesVirtual Samples Displacement
Vectors
Originals Morphs
Twist (5deg)+ Taper- Push
+ Spherify (10%)
Push +Twist (10 deg)
+Scale (1.2)
Original
Global Morphing Operators
Morphs
Physical Modeling
(14)
University of WashingtonElectrical Engineering
Task 1: Recognizing Single Task 1: Recognizing Single Objects (1)Objects (1)
No. Shape classes: 9.No. Shape classes: 9. Training set size: 400 meshes.Training set size: 400 meshes. Testing set size: 200 meshes.Testing set size: 200 meshes. No. Experiments: 1960.No. Experiments: 1960. No. Component detectors:3.No. Component detectors:3. No. Symbolic signature detectors: 1.No. Symbolic signature detectors: 1. Numeric signature size: 40x40.Numeric signature size: 40x40. Symbolic signature size: 20x20.Symbolic signature size: 20x20. No clutter and occlusion.No clutter and occlusion.
Task 1: Recognizing Single Task 1: Recognizing Single Objects (2)Objects (2)
Snowman: 93%.Snowman: 93%. Rabbit: 92%.Rabbit: 92%. Dog: 89%.Dog: 89%. Cat: 85.5%.Cat: 85.5%. Cow: 92%.Cow: 92%. Bear: 94%.Bear: 94%. Horse: 92.7%.Horse: 92.7%.
Human head: Human head: 97.7%.97.7%.
Human face: 76%.Human face: 76%.
Recognition rates (true positives)Recognition rates (true positives)(No clutter, no occlusion, complete models)(No clutter, no occlusion, complete models)
Tasks 2-3: Recognition In Tasks 2-3: Recognition In Complex Scenes (1)Complex Scenes (1)
No. Shape classes: 3.No. Shape classes: 3. Training set size: 400 meshes.Training set size: 400 meshes. Testing set size: 200 meshes.Testing set size: 200 meshes. No. Experiments: 1200.No. Experiments: 1200. No. Component detectors:3.No. Component detectors:3. No. Symbolic signature detectors: 1.No. Symbolic signature detectors: 1. Numeric signature size: 40x40.Numeric signature size: 40x40. Symbolic signature size: 20x20.Symbolic signature size: 20x20. T2 – low clutter and occlusion.T2 – low clutter and occlusion.
Task 2-3: Recognition in Task 2-3: Recognition in Complex Scenes (2)Complex Scenes (2)
ShapeShape
ClassClassTrueTrue
PositivePositivess
FalseFalse
PositivePositivess
True True
PositivePositivess
FalseFalse
PositivePositivess
SnowmeSnowmenn
91%91% 31%31% 87.5%87.5% 28%28%
RabbitRabbit 90.2%90.2% 27.6%27.6% 84.3%84.3% 24%24%
DogDog 89.6%89.6% 34.6%34.6% 88.12%88.12% 22.1%22.1%
Task 2Task 2 Task 3Task 3
Task 2-3: Recognition in Task 2-3: Recognition in Complex Scenes (3)Complex Scenes (3)
Main Contributions (1)Main Contributions (1)
A novel A novel symbolic signature symbolic signature representationrepresentation of deformable shapes that of deformable shapes that is robust to intra-class variability and is robust to intra-class variability and missing information, as opposed to a missing information, as opposed to a numeric representationnumeric representation which is often which is often tied to a specific shape.tied to a specific shape.
A novel A novel kernel functionkernel function for quantifying for quantifying symbolic signature similarities. symbolic signature similarities.
Main Contributions (2)Main Contributions (2)
A A region growingregion growing algorithm for learning algorithm for learning shape class components. shape class components.
A novel A novel architecture of classifiersarchitecture of classifiers for for abstracting the geometry of a shape class.abstracting the geometry of a shape class.
A validation of our methodology in a set A validation of our methodology in a set of of large scalelarge scale recognition and recognition and classification experiments aimed at classification experiments aimed at applications in scene analysis and medical applications in scene analysis and medical diagnosis.diagnosis.