Robotica e Sensor Fusion per i Sistemi Meccatronici Object Detection with Superquadrics

M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion

Robotica e Sensor Fusion per i Sistemi Meccatronici

Object Detection with Superquadrics

Prof. Mariolino De Cecco, Dr. Ilya Afanasyev, Ing. Nicolo Biasi

Department of Structural Mechanical Engineering, University of Trento

Email: [email protected]

[email protected]

http://www.mariolinodececco.altervista.org/


2/21

The block diagram of recognition task:The block diagram of recognition task:

02/04/2012

Object detection and localization

Object Measurement system

Data

Fitting Algorithm

Pre-processing

Math. model

- Dimensions;A-priori know:

- Features (shape, curvature, etc.)

- Superquadrics, etc.

- Camera, laser- Kinect- Multicamera system

- Point clouds- Images, video

- Eliminating the background;- Changing the contrast, size, etc.

- Optimization, iterations, etc.;

- Criteria of quality.

- RANSAC;

Results- Transformation matrix, angles, distance, accuracy, etc.


Examples of Superquadrics


[1]



Wireframes of Superquadrics

[1]


Examples of Superellipsoids

Examples of Superellipsoids

[4]

Superellipsoids can model spheres, cylinders, parallelepipeds and shapes in between. Modeling capabilities can be enhanced by tapering, bending and making cavities.


About Superquadrics

About SuperquadricsThe term of Superquadrics was defined by Alan Barr in 1981 [2]. Superquadrics are a flexible family of 3D parametric objects, useful for geometric modeling. By adjusting a relatively few number of parameters, a large variety of shapes may be obtained. A particularly attractive feature of superquadrics is their simple mathematical representation.

Superquadrics are used as primitives for shape representation and play the role of prototypical parts and can be further deformed and glued together into realistic looking models.

[9]


Classification of Superquadrics [2,9]

Classification of Superquadrics

a) Superellipsoids.

b) Superhyperboloids of one piece.

c) Superhyperboloids of two pieces.

d) Supertoroids.

a) b) c)

d)[9]


Definition of Spherical Products

Definition of Spherical Products

Example. For two 2D curves: circle and parabola, the spherical product is 3D paraboloid.

For two 2x1 vectors [a b]’ and [c d]’ the spherical product is 3x1 vector for which:


Spherical Products [9]

Spherical ProductsA 3D surface can be obtained by the spherical product of two 2D curves [2]. The spherical product is defined to operate on two 2D curves. A unit sphere is produced by a spherical product of a circle h(ω) horizontally and a half circle m(η) vertically.


Superellipsoids

The equation of Superellipse is

Spherical Products for Superellipsoids

and in parametric form:

Superellipsoids can be obtained by a spherical products of a pair of such superellipses:

The implicit equation is:

- are parameters of shape squareness;

a1 a2 a3 - parameters of Superellipsoid sizes.

1 2where


Superhyperboloids of one piece

Spherical Products for Superhyperboloids of one piece

Superhyperboloids of one piece can be obtained by a spherical products of a hyperboloid and a superellipse:




1 2where


Superhyperboloids of two pieces

Spherical Products for Superhyperboloids of two pieces

Superhyperboloids of two pieces can be obtained by a spherical products of a pair of such hyperboloids:




1 2where


Supertoroids

Spherical Products for Supertoroids

Supertoroids can be obtained by a spherical products of the following surface vectors:



- parameters of Superellipsoid sizes.

1 2where


15/21

Modelling Superquadrics in MATLAB:Modelling Superquadrics in MATLAB:- - Demo-function: Demo-function: xpquadxpquad - superquadrics plotting demonstration - superquadrics plotting demonstration..- ezmesh - ezmesh and and ezsurfezsurf functions (see, Superquadrics_visualization.m). functions (see, Superquadrics_visualization.m).

02/04/2012

Superquadrics in MATLAB

%---------------------------------------------------------------% a1 = 4; a2 = 2; a3 = 1; eps1 = 1; eps2 = 1; figure('Name','Superellipsoids')ezmesh('4*sin(u)*cos(v)', '2*sin(u)*sin(v)', 'cos(u)', [0, 2*pi], [0, 2*pi], 20)colormap([0 0 1])hold onrotate3d on%---------------------------------------------------------------


What a1, a2 and a3 mean?

05/04/2011 16/20

a1, a2 and a3 – are parameters of parallelepiped’s semi-sides.

The parameters of shape squareness for parallelepiped are: ε1 = ε1 = 0.1

If parallelepiped has dimensions: 20 x 30 x 10 cm, it means that a1 = 10, a2 = 15 and a3 = 5.


What ε1 and ε2 mean?

ε1 = 0.1 ε1 = 1 ε1 = 2

ε2 = 0.1

ε2 = 1

ε2 = 2

[9]

05/04/2011 17/20

ε1 and ε2 – are parameters of shape squareness.


Superellipsoids shapes varying from ε1, ε2

05/04/2011 18/20

ε1 = 3

ε2 = 1

ε1 = 1

ε2 = 3

ε1 = 3

ε2 = 3


Vector r(η,ω) sweeps out a closed surface in space when η,ω change in the given intervals:

η

ω

x

y

z

r(η,ω)

Plotting Superellipsoids in spherical coordinates

η,ω – independent parameters (latitude and longitude angles) of vector r(η,ω) expressed in spherical coordinates.

2

2

r ( )

a1cos ( )1

cos ( )2

a2cos ( )1

sin ( )2

a3sin ( )1

a1

a2

a3

Superellipsoids


x,y – independent parameters (Cartesian coordinates of SQ) are used to obtain z.

f x y z( )x

a1

2

2y

a2

2

2

2

1

z

a3

2

1

Use the implicit equation in Cartesian coordinates, considering

f(x,y,z) = 1-a1 ≤ x ≤ a1

-a2 ≤ y ≤ a2

05/04/2011 20/20

x’

x

y

z

y’

z’

z=NaN

y=a2

x=a1

Plotting Superellipsoids in Cartesian coordinates

The implicit form is important for the recovery of Superquadrics and testing for intersections, while the explicit form is more suitable for scene reconstruction and rendering.


Warning: complex numbers in SQ equation

1. If ε1 or ε2 < 1 and cos or sin of angles ω or η < 0, then vector r(η,ω) has complex values. To escape them, it should be used signum-function of sin or cos and absolute values of the vector components.

2

2

2. Analogically if x or y < 1 and ε1 > 1, the function f(x,y,z) willl have the complex values of z. To overcome it, use the f(x,y,z) in power of exponent ε1.

f x y z( )x

a1

2

2y

a2

2

2

2

1

z

a3

2

1

f(x,y,z)ε1 = 1

21/20

05/04/2011


Rotation and translation of SQ

Elevation

Azimuth

x

y

z

zW

xW

yW

T – transformation matrix.n – amounts of points in SQ surface.

SQ – coordinates of points of SQ surface. Pw – coordinates of points of rotated SQ surface.

xW, yW, zW – world system of coordinates (with center in viewpoint).

05/04/2011 22/20


Elevation

Azimuth

x

y

z

zW

xW

yW

Rotation and translation of SQ

px

az,el,px,py,pz,x,y,z – are given; xW,yW,zW – should be found.

T SQ Pw

05/04/2011 23/20


Applications with SuperquadricsSuperquadrics have been employed in computer vision and robotics problems related to object recognition. Superquadrics can be used for

1. Object recognition by fitting geometric shapes to 3D sensor data obtained by a robot.

Shape reconstruction is a low level process where sensor data is interpreted to regenerate objects in a scene making as few assumptions as possible about the objects. Object recognition is a higher level process whose goal is to abstract from the detailed data in order to characterize objects in a scene.

2. Scene reconstruction and recognition. Rendering.

In order to fit a superquadric to a surface region, 11 parameters must be determined: three extent parameters (a1, a2, a3), two shape parameters (ε1 and ε2), three translation parameters, and three rotation parameters.

Applications with Superquadrics


Definition of Superquadrics

Figures from Superquadrics

[3]


Reconstruction of complex object [6]

Applications with SQ



05/04/2011 26/20


Reconstruction of multiple objects [9]

Applications with SQ

Reconstruction of multiple objects [9]

05/04/2011 27/20


28/21

Dimensions of EuroPalletDimensions of EuroPallet

02/04/2012

Task 1: To model Europallet in SQ


It should be created It should be created SQ model of palletSQ model of pallet

29/21

As an example of a Pallet model

02/04/2012

1000

0100

0010

001 d

T SQcSQr

Transformation: SQc – SQr

z

xy

z

xy

z

xydd

SQc SQrSQl

1000

0100

0010

001 d

T SQcSQl

Transformation: SQc – SQl


30/21

Objects like an apple, a bottle, a chair, a tin of beerObjects like an apple, a bottle, a chair, a tin of beer

02/04/2012

Task 2: To model objects by deformed SQ


Links1. A. Skowronski, J. Feldman. Superquadrics. cs557 Project, McGill University.

http://www.skowronski.ca/andrew/school/557/start.html

2. Barr A.H. Superquadrics and Angle-Preserving Transformations. IEEE Computer Graphics and Applications, 1, 11-22. 1981.

3. Chevalier L., etc. Segmentation and superquadric modeling of 3D objects. Journal of WSCG, V.11 (1), 2003. ISSN 1213-6972.

4. Kindlmann G. Superquadric Tensor Glyphs. EUROGRAPHICS. IEEE TCVG Symposium on Visualization (2004). Pages 8.

5. Solina F. and Bajcsy R. Recovery of parametric models from range images: The case for superquadrics with global deformations. // IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-12(2):131--147, 1990.

6. Chella A. and Pirrone R. A Neural Architecture for Segmentation and Modeling of Range Data. // 10 pages.

7. Leonardis A., Jaklic A., and Solina F. Superquadrics for Segmenting and Modeling Range Data. // IEEE Transactions On Pattern Analysis And Machine Intelligence, vol. 19, no. 11, 1997.

8. Bhabhrawala T., Krovi V., Mendel F. and Govindaraju V. Extended Superquadrics. // Technical Report. New York, 2007. 93 pages.

9. Jaklic Ales, Leonardis Ales, Solina Franc. Segmentation and Recovery of Superquadrics. // Computational imaging and vision 20, Kluwer, Dordrecht, 2000.

Grazie per attenzione!!

Documents

Robotica e Sensor Fusion per i Sistemi Meccatronici Object Detection with Superquadrics