EE 576 - Object - Scene Representationisl.ee.boun.edu.tr/courses/ee576/lectures/sunum/repres.pdf · Object–SceneAnalysis Introduction Model-BasedObjectRepresentation GraphRepresentation

OutlineObject – Scene Analysis

EE 576 - Object - Scene Representation

H.I. Bozma

Electric Electronic Engineering

Bogazici University

March 28, 2012

H.I. Bozma EE 576 - Object - Scene Representation


Object – Scene AnalysisIntroductionModel-Based Object RepresentationGraph Representation



IntroductionModel-Based Object RepresentationGraph Representation

Scene Understanding

Scene analyis (scene understanding) is the ability to make usefuldescriptions of the given scene.

◮ Identify objects and relationships between them

◮ Locate a certain object

◮ Make measurements for inspection




Problems in Scene Analysis

◮ Representation

◮ Occlusion

◮ Image complexity

◮ Scene complexity

◮ A priori knowledge is essential




Representation

◮ Levels of abstraction◮ Specific objects◮ Generic objects◮ Functional representations

◮ Properties of objects: Shape, color, texture, function

◮ Shape representation: Collection of points, boundaries,surfaces, volumes, collection of parts




Projection and Occlusion

Projection and occlusion – create difficulties.Scenes 3-D – images 2-D.Projected shapes – Depend on the viewpoint.Objects occlude each other → Only parts of objects visible →

Problems in segmentation.




Complexity

◮ Images are complex◮ Nature – Monocular, stereo, range◮ Quality – nature

◮ Scenes are complex◮ Large number of objects◮ Occlusion◮ Similar objects◮ Complex shapes◮ Complex patterns◮ Illumination




General Approach

◮ Choose a representation model

◮ Find instances of known models in the scene

◮ Estimate their pose




Complexity

◮ The number of models in the database

◮ Complexity of the image

◮ Complexity of the scene




Matching Models – General Approaches

◮ Template matching

◮ Correlate a template with the image◮ Invariant to translation, sensitive to rotation, scaling and

viewpoint◮ Useful for 2D binary patterns (character recognition)

◮ Feature vectors

◮ Describe by features – Area, moments, Fourier coefficients◮ Solve segmentation◮ Viewpoint invariance is hard

◮ Features and the relationships between them




Representation and Recognition Paradigm

◮ Could the image be a projection of the model?

◮ Match part of a scene with a model → Search for best fit◮ Estimate object pose◮ Project model and verify

◮ What types of features to match and how?◮ Feature matching◮ Structured description matching




Recognition Paradigm

Figure: Recognition paradigm




Feature Matching

◮ Match image and model features → Search for best fit

◮ Check for consistency using constraints that depend onrepresentatin




Feature Matching

Figure: Feature matching




Search Tree: Breadth-First

Figure: Breath-first search of a match tree.




Perceptual Grouping

Perceptual grouping: Organization of image features into higherlevel feature

◮ Descriptive – Make geometrical structure explicit

◮ Hierarchical

◮ Invariant – if possible




Perceptual grouping

Figure: Perceptual organization




Perceptual Organization Criteria

◮ Similarity – Features (Lines – Bending energy)

◮ Proximity

◮ Continuation

◮ Symmetry

◮ Closure

◮ Familiarity




Bending Energy

Minimum bending energy - as measured by curvature.





Perceptual grouping – Bending energy

→





Representation By Parts

◮ One approach to structural shape representation is graphrepresentation

◮ A graph consists of two entities: nodes P and edges E (P)between nodes.

◮ Properties of nodes and edges in the model database.

Some examples are:

◮ Line drawings consisting of vertices (nodes) and lines (edges)

◮ Surfaces (nodes) and their relationships (edges)

◮ Complex shape represented by parts (nodes) and therelationships (edges)




Recognition by Structure Matching

◮ Here, the model is based on a graph that represents theobjects’ parts and their relationships.

◮ Given an incoming image, a similar model is constructed andmatched with those in the model database.




Graph Matching

Given G = (P ,E (P),R(P)) and G′ = (P ′

,E′(P ′),R ′(P ′)

◮ Two nodes n ∈ P and n′∈ P

′ form an assignment ifR(n) = R

′(n′)

◮ Two assignments (n1, n′

1) and (n2, n′

2) are compatible ifE (n1, n2) = E (n′1, n

′

2)

◮ Isomorphism: Compatibility for all nodes

◮ Isomorphism: Compabilility for a subset of nodes




Graph Isomorphism

Finding an isomorphism is NP-hard:




Graph Isomorphism – Ex 2

Figure: Graph Matching - IIH.I. Bozma EE 576 - Object - Scene Representation



Aspect Graphs

Aspect graph – A series of two-dimensional views (nodes) of aknown object that are related to each other (edges) via 3Dtransformations.

◮ Generate the aspect graph

◮ Maintain them in some convenient representation in storage,

◮ Match one or more two-dimensional views of an unknownobject against the stored views of the known object,

Recognition problem → Transformed to a series of two-dimensionalmatching problems.




Varying viewpoints

Figure: Different views of an object.H.I. Bozma EE 576 - Object - Scene Representation



Varying Viewpoints

Figure: Different views of a L-shaped object.H.I. Bozma EE 576 - Object - Scene Representation



Aspect Graph

Figure: Aspect graph of an a L-shaped shaped object.H.I. Bozma EE 576 - Object - Scene Representation

Documents

EE 576 - Object - Scene Representationisl.ee.boun.edu.tr/courses/ee576/lectures/sunum/repres.pdf · Object–SceneAnalysis Introduction Model-BasedObjectRepresentation GraphRepresentation