Autonomous Learning of Object Models on Mobile Robots

Xiang Li

Ph.D. student supervised by Dr. Mohan SridharanStochastic Estimation and Autonomous Robotics Lab (SEARL)

Department of Computer ScienceTexas Tech University

October 23, 2012

1

Object Recognition

2

Learn Object Modelcolor, texture or shape

Test

match

Challenges:• How to identify ROI in the image (Region Of Interest) ?

• What features to extract in ROI (Object model) ?

• Efficient Implementation.

ROI

Related Work

• Object model O. Bjorn, PAMI12; R. Fergus, CVPR03; P. Felzenszwalb, IJCV04; N.E.K. Roman,

AR10.

• Mobile robot

C. Guo, ICRA11; M. Sridharan, IROS07; D. Parikh, PAMI12; J. Hoey, CVIU10.

• Local image feature

D. Lowe, IJCV05; J. Matas, BMVC02; S. Park, IROS09; M. Calonder, ECCV10.

3

Motivation

4

• Goal

Learn and recognize objects autonomously in dynamic environments.

• Our Work Identify ROIs autonomously based on a limited number of images with moving objects. Build probabilistic object models using the complementary properties of different visual cues. Fuse the information by generative model and energy minimization algorithm.

Autonomously Learning• Supervised Learning

Images with the labeled regions

• Unsupervised Learning Images without any labeled regions

Track and cluster local image gradient features [128D vector] A short sequence of images (motion cue)

5

ROI

t t+1

ROI

Object Model• Given ROI (autonomous or manual)• Use the complementary properties of different visual cues

6

Motivation of SCV and Undirected Graph

• The difference between correct and incorrect match The spatial arrangements of local features The connection between the local features

7

• Match by Nearest Neighbor algorithm(shortest Euclidean distance).

correct

Incorrect

SCV from gradient features

• The individual gradient features may not be unique.• The spatial arrangement of local gradient features corresponding to a

specific object is difficult to duplicate.

8

Connection Potentials

• Connection potential is computed as the color distribution of pixels between gradient features in the image ROI.

9

• Build an undirected graph of connection potentials to model the neighborhood relationships between gradient features.

Parts from image segments

• Considers the arrangement of object parts made up of image segments.

• Pixels within a part have similar values, while pixels in neighboring parts have dissimilar values.

10

Color-based Representation• Computes the distance between every pair of pdfs and models the

distribution of distances as a Gaussian.

11

Second order color distribution statistics Color histogram (pdf)

… …

Local context from image segments

• Probabilistic mixture models

• Relative positions (on, under, beside)

12

Information Fusion

13

• Generative model considers the relationship between the components

• Energy minimization algorithm Identifies the ROI for recognizing the stationary objects

Robot Platforms: Erratic

• 1.6GHz Core2 Duo CPU

• 2 cameras (monocular & stereo)

• Laser range finder

• 640 × 480 Resolution

• Wi-Fi

• On board computation

14

Experimental Trial

15

Test Image Match Probabilities Net Match

Experimental Trial (Cont)

16

Test Image Match Probabilities Net Match

Object Categories

17

The Classification Accuracy

18

Conclusions• Mobile robot can identify interesting objects based on motion

cues, and autonomously and efficiently learn object models that exploit the complementary properties of appearance-based and contextual visual cues.

• Exploiting the complementary properties of these visual cues enables the robot to use generative model and energy minimization algorithms to reliably and efficiently recognize the learned.

19

Future Work

• Consider image sequences with multiple moving objects.

• Add Shape representation into object model.

• Extend to a team of robots collaborating in dynamic environments.

20

Q & A

21

Convex Hull

• The convex hull of a set of points is the smallest convex set that contains the points.

• Quickhull algorithm1 computes the convex hull.

1. Barber, C.B., Dobkin, D.P., and Huhdanpaa, H.T., “The Quickhull algorithm for convex hulls”, ACM Trans. on Mathematical Software, 22(4):469-483, Dec 1996, http://www.qhull.org

Gamma distribution

• Context Probability

• Context Probability Using Gamma

Matched Not Matched

context 0.28±0.15 0.03±0.03

Other components 0.7 0.3

Matched Not Matched

context 0.5±0.25 0.09±0.10

Other components 0.7 0.3

Experiment Example• Learned robot model (Corridor)

• Testing

Corridor(1)Prob = 0.89

Corridor(2)Prob = 0.77

OfficeProb = 0.12

Generative Model

• Randomly generating observable data• Typically given some hidden parameters• Specifies a joint probability distribution over

observation and label sequences.

25

Generative model (Cont)

26