Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa [email protected] anna

Research in Intelligent Mobile Robotics (and related topics)Part 1: Navigation and Vision

Anna Helena Reali [email protected] www.pcs.usp.br/~anna

Laboratório de Técnicas Inteligentes Escola Politécnica da Universidade de São Paulo

Carlos Henrique Costa [email protected] www.comp.ita.br/~carlos

Divisão de Ciência da Computação Instituto Tecnológico de Aeronáutica

MultiBot - Meeting #1, Lisboa 2003 - part I 2

Preface

This is a two-part talk about the research on Intelligent Mobile Robotics and related topics at: LTI – USP (Laboratório de Técnicas Inteligentes –

Universidade de São Paulo, Brazil) NCROMA-ITA (Laboratório de Navegação e Controle

de Robôs Móveis Autônomos – Instituto Tecnológico de Aeronáutica, Brazil).

These research groups are involved in the MultiBot cooperation project CAPES/GRICES with ISLab-IST.


LTI - EPUSP

Prof. Anna Reali 5 PhD Students

Alexandre Simões*, Reinaldo Bianchi, Valdinei Silva*, Valguima Odakura, Waldemar Bonventi.

3 Master Students Alexandre Cunha*, Antônio Selvatici, Luiz Carlos Maia

3 Undergraduate Students Rafael Pacheco, Márcio Seixas, Júlio Kawai

2 Final Course Projects


NCROMA – ITA

Prof. Carlos Ribeiro 1 PhD Student

Letícia Friske

5 Master Students Luís Almeida, Ricardo Maia, Juliano Pereira,

Esther Colombini*, Celeny Alves*

2 Undergraduate Students Lucas Gabrielli, Fábio Miranda


Intelligent Mobile Robotics

Part 1:Navigation• Map building• Localization

Perception• Computer Vision

Part II:Learning

Contents


Map Building

Our goal: Test map building algorithms in real robots fast enough? precise enough? ok for learning applications (e.g. path

learning)?


Map BuildingSildomar Takahashi, Carlos RibeiroRoberto Barra, Ricardo Domenecci, Anna Reali

“Efficient Learning of Variable-Resolution Cognitive Maps for Autonomous Indoor Navigation”.

Arleo, Millan e Floreano, IEEE-SMC, 1999. Advantages:

Complete algorithmic description Simple structure

Limitations: Assumes structure (orthogonal obstacles / walls) Reliance on dead-reckoning (but can be adapted to

more sophisticated localization)


The Basic Algorithm

1. Explores environment;

2. Once an obstacle is detected:

1. Determines obstacle frontiers either via: An a priori sensor model

A pre-trained neural net

2. Includes obstacle in the global map;

3. Defines new partition to explore. If there is none, END. Else finds route to new partition.

4. Executes route and explores new partition. Once an obstacle is detected, Step 2. Else, Step 1.


Detection of obstacle frontiers

Robô

Células Ocupadas

RetaCalculada

• Either a priori model or neural net model• Integration over time• Straight-line adjustment and correction (according to a priori actuator model)

0 y

x

My

Mx


Very “Scientific” Set-up

Walls Obstacles

Robot

3 x 3,5 m


Global Maps: Magellan, Neural net model

Map 1

Map 2

Map 3


Global Map: Pioneer, a priori model + straight-line model-based correction


Conclusions Tested algorithm (possibly with some

modifications) is a good compromise efficiency/precision for realistic applications: fast yet fairly accurate.

Next steps: Studies on simultaneous localization and

mapping (SLAM algorithms). Valguima Odakura (Anna Reali): SLAM based

on visual landmarks. Fabio Miranda (Carlos Ribeiro): Bayesian

landmark learning. Techniques for map building acceleration.


Markov LocalizationLuís Almeida, Carlos RibeiroJúlio Kawai, Anna Reali

Position estimation based on Bayesian update: Belief update based on sensor info Belief update based on action info

Sensor/actuator models and initial belief distribution: arbitrary.

Simple to implement.Computationally costly (Monte Carlo

implementation – particle filters – is a possible fix).


Markov Localization

X

p (x )

b )

X

p (x )

d )

X

p (x )

h )

X a)1

X c)1

X e)1 X2

X

p (x )

f)

X g )2

Probabilistic position grid

Action Model

Sensor Model Markov State

Estimator

pt

st

at

pt+1

odometerssonars


Monte Carlo Localization + GA OptimizationLuís Almeida, Carlos Ribeiro

Standard Markov update (over set of particles)



MC MC MCGA GA

GA on population of particles (fitness as combination of belief / particle cluster distribution)

GA on population of particles (fitness as combination of belief / particle cluster distribution)

• Basic idea: use GA to create a better set of particles for next MC update.

• Initial results: ok (in need of statistical validation).


Next Steps

Validation of GA approach. Better sensor and actuator models. Implementation in a real robot. Literature on Monte Carlo methods

(applications on signal detection and tracking): many variations to be tried...


Computational Vision

Image Segmentation Using Color Classification Using Background Model Using Optical Flow Based on Binocular Stereo Vision


Color Classification - I

Using threshold values: In the color representation space

Neural Network – MLP + backpropagation alg.:Alexandre Simões, Anna Reali


Derived Application:Alexandre Simões, Anna Reali

C1 C2 C3 C4 C5

Orange Classifier

Branco

. . .

Verde Claro

Verde Escuro

Amarelo

Laranja Claro

Laranja Escuro

R

G

B

Danificado

Orange Classifier - CEAGESP, SP


For number of clusters = 2 to Cmax, do: Apply FCM-GK (non-supervised fuzzy classifier)

to RGB image; Calculate the ratio c/s for each cluster set:

c = Cluster dimension/number of members s = Separation among clusters

Choose the cluster set, based on c/s. Show color classification result for the best

cluster set.

Non-supervised iterative fuzzy color classificationWaldemar Bonventi, Anna Reali


An example: soccer

The best cluster set 6 clusters


Another example: Rio de Janeiro

The best cluster set 3 clusters





Background Model - I

Background subtraction Thresholding the error between an estimate of

the image without moving objects – M(C) – and the current image:

Model can not adapt to environment changes!

M(C)Current Image


Background Model – IIMárcio Seixas, Anna Reali

Time-Adaptive, Per-Pixel Mixture-of-Gaussians: Time series of observations at a given pixel (its color) is

modeled by a mixture-of-gaussians. Based on the persistence and the variance of each of the

gaussians of the mixture, it is determined which gaussians may correspond to background colors.

Hypothesis: gaussian distributions with low variance and high persistence correspond to background model.

Per-pixel models are updated as new observations are obtained (according to a learning rate).

It is capable of dealing with long-term scene changes (e.g. lighting changes)!


Derived Application:platform occupancy

Fixedmodel M(C):

Original:

Original – M(C):

Adaptive Model:

Terminal Rodoviário de Santo Amaro TRENDS & Prefeitura de São PauloMárcio Seixas, Anna Reali





Optical Flow - idea


Robot (with camera) navigating in a stationary scenario. Calculation of the optical-flow divergent to estimate the time-

to-crash value in order to avoid collisions with obstacles. We are now investigating a robust method to directly

calculate the per-pixel time-to-crash value:

Vision-based robotic behavior:Antonio Selvatici, Anna Reali

Gray levels: near bright; far darkBlack: unknown distance

Original sequence Pixel time-to-crash Filtered values


Derived Application: monitoring of underground rail tracksLuiz Maia, Anna Reali

ALSTOM & Metrô de São Paulo





Distance-Map Calculation1. Calibration [Zhang, ICCV 99]

2. Matching – blob coloring+centroid+correlation

3. Triangulation Segmentation: based on color + distance-map.

Binocular Stereo VisionRafael Pacheco, Anna Reali


Derived Application:Outdoors Measurement

TRENDS & Prefeiturade São PauloRafael Pacheco, A. Reali


Conclusions In CV, we are now investigating:

Automatic learning of fuzzy color classifiers – Waldemar Bonventi, LTI;

A framework for high-level feedback to adaptive, per-pixel, mixture-of-gaussian background models – Márcio Seixas, LTI;

Mathematical formulation for direct and robustly calculate the per-pixel, time-to-crash values, considering a moving observer in a stationary scenario – Antonio Selvatici, LTI;

Distributed, real-time approach to calculate the optical flow, considering a stationary observer in a dynamic scenario – Luiz Maia, LTI.

Documents

Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa [email protected] anna