54
Yuttana Suttasupa Advisor: Asst.Prof. Dr.Attawith Sudsang 3D SLAM for Omni- directional Camera

3D SLAM for Omni-directional Camera

  • Upload
    oswald

  • View
    96

  • Download
    0

Embed Size (px)

DESCRIPTION

3D SLAM for Omni-directional Camera. Yuttana Suttasupa Advisor: Asst.Prof . Dr.Attawith Sudsang. Outline. Introduction Localization, Mapping, SLAM, SLAM Application Related Work Vision SLAM, SLAM with Omni-directional Camera Our Problem - PowerPoint PPT Presentation

Citation preview

Page 1: 3D SLAM for Omni-directional Camera

Yuttana SuttasupaAdvisor: Asst.Prof. Dr.Attawith Sudsang

3D SLAM for Omni-directional Camera

Page 2: 3D SLAM for Omni-directional Camera

2

Outline Introduction

Localization, Mapping, SLAM, SLAM Application

Related Work Vision SLAM, SLAM with Omni-directional Camera

Our Problem Challenge of problem, Propose method, SLAM

Algorithm

Etc. Scopes, Work plan

Page 3: 3D SLAM for Omni-directional Camera

3

Localization Robot can estimate its location with respects to

landmarks in an environment

Page 4: 3D SLAM for Omni-directional Camera

4

Mapping Robot can reconstruct the position of landmarks

that its encounter in an environment

Page 5: 3D SLAM for Omni-directional Camera

5

Simultaneous localization and mapping (SLAM) Robot build up a map and localize itself

simultaneously while traversing in an unknown environment

[Paul Newman, 06]

Page 6: 3D SLAM for Omni-directional Camera

6

Objective Introduce SLAM method for a hand-held omni-

directional Camera moving freely in an unknown environment

Algorithm can reconstruct 3D camera path and 3D environment map

Page 7: 3D SLAM for Omni-directional Camera

7

SLAM Challenges (Why is SLAM hard?) “Chicken and Egg” problem

Robot needs map to localize itself Robot needs to know its location to reconstruct

landmark positions Uncertainty problem

Sensor noise Data association

Page 8: 3D SLAM for Omni-directional Camera

8

SLAM SLAM Problem

How can a robot localize its own position and build up a map in an unknown environment

SLAM Algorithm A procedure to solve a SLAM Problem Using a probabilistic approach to solve Find an appropriate representation for the observation

model and motion model SLAM Application

How to applied the observation model in our interest environment

What is a motion model for our interest robot It’s our work

Page 9: 3D SLAM for Omni-directional Camera

9

Simultaneous localization and mapping (SLAM)

Page 10: 3D SLAM for Omni-directional Camera

10

SLAM Application Indoor Environment Outdoor Environment At Home Under water UAV Navigation

[Ahn et al.,07; Davison et al,06; ...]

[Miro et al.,06; Newman, 07; Han et al., 07; ...]

[Choi et al.,06; Ahn et al,06; Motard et al.,07]

[Williams et al.,01; Ribas et al.,06]

[Jonghyuk Kim et al.,07]

Page 11: 3D SLAM for Omni-directional Camera

11

Sensor

Sonar Laser range finder Omni-directional CameraVideo Camera

Sensor Price Accuracy Data typeSonar cheap low rangeLaser Range Finder

expensive high range & bearing

Camera vary (up to image quality)

moderate bearing

Page 12: 3D SLAM for Omni-directional Camera

12

Vision SLAM Stereo Vision Based SLAM

[Thanh et al.,06; Schleicher et al.,06; Schleicher et al.,07; Lemaire et al.,07; Marzorati et al.,07; Han et al.,07, ...]

Page 13: 3D SLAM for Omni-directional Camera

13

Vision SLAM Monocular SLAM

[Davison et al.,03; Eade et al.,06; Sunderhauf et al.,07; ...]

Page 14: 3D SLAM for Omni-directional Camera

14

Omni-direction Camera Advantage

Large field of view (360° field of view) Real-time information @ 29.97Hz (NTSC) Vision data with color information Inexpensive

Disadvantage low resolution compare to the FOV

Page 15: 3D SLAM for Omni-directional Camera

15

Omni-direction Camera

Page 16: 3D SLAM for Omni-directional Camera

16

SLAM with Omni-directional Camera SLAM with Omni-directional Stereo Vision Sensor [Kim et al., 03] Visual SLAM by Single-Camera Catadioptric Stereo [Kim et al., 06]

Page 17: 3D SLAM for Omni-directional Camera

17

SLAM with Omni-directional Camera Localization of mobile robots with omnidirectional vision

using particle filter and iterative SIFT [Tamimi et al., 05] Localization for Mobile Robots using Panoramic Vision,

Local Features and Particle Filter [Andreasson et al., 05]

Page 18: 3D SLAM for Omni-directional Camera

18

SLAM with Omni-directional Camera Incremental Topological Mapping Using Omnidirectional

Vision [Valgren et al., 06] Appearance-based SLAM with Map Loop Closing Using

an Omnidirectional Camera [Saedan et al., 07]

Page 19: 3D SLAM for Omni-directional Camera

19

SLAM with Omni-directional Camera SLAM in Indoor Environments using Omni-directional

Vertical and Horizontal Line Features [Kim et al., 08]

Page 20: 3D SLAM for Omni-directional Camera

20

Our Problem Propose SLAM method for a hand-held omni-

directional Camera Omni-directional camera can move freely in an

unknown indoor environment Reconstruct 3D camera path and 3D

environment map (landmark-based) No need any initial information or predefine data

Page 21: 3D SLAM for Omni-directional Camera

21

Challenge Irregular Sensor

Uncommon sensor’s measurement model What features should we use to be a measurement

Insufficient information Bearing-only data Unpredictable camera trajectory Don’t have any initial information

A high dimensional state Need to estimate 3D camera path Need to estimate 3D environment map

Page 22: 3D SLAM for Omni-directional Camera

Proposed Method

Page 23: 3D SLAM for Omni-directional Camera

23

Localization Concept

x

bel(x)

x

bel(x)

x

bel(x)

x

bel(x)

Initial state

Measurement-update

Measurement-update

Time-update

observation

observation

Page 24: 3D SLAM for Omni-directional Camera

24

SLAM Algorithm Probabilistic SLAM [Smith and Cheeseman, 86]

The probability distribution of robot state and landmark locations

The observation model

The motion model

Page 25: 3D SLAM for Omni-directional Camera

25

SLAM Algorithm SLAM recursive algorithm

Time-update

Measurement Update

Page 26: 3D SLAM for Omni-directional Camera

26

Using SLAM in our problem Use EKF SLAM to solve our SLAM problem

Find an observation modelhow to measurement landmarkshow to detect features from an omni-image

Find a motion modeldetermine how a camera move

Find a state representationhow to represent a camera statehow to represent landmarks state

[Moutarlier and Chatila, 89]

Page 27: 3D SLAM for Omni-directional Camera

27

An observation model Detect features from an

omni-image Point features, Line features

Features association How features associate with

landmarks

Feature measurements Observation model

landmark

camera

Page 28: 3D SLAM for Omni-directional Camera

28

A motion model Camera can move freely in an environment

Don’t know camera motion model Cannot predict camera trajectory Too many state variables to estimate with SLAM only

May need to pre-estimate camera state Using pre-estimate camera state to predict camera

state

time k time k + m

camera

landmark

Page 29: 3D SLAM for Omni-directional Camera

29

Result evaluation Localization evaluation

Using ground truth data Using global localization

Mapping evaluation Using ground truth data from structural environment

Page 30: 3D SLAM for Omni-directional Camera

30

Scope of the research Develop SLAM method for an omni-directional

Camera Develop an algorithm to detect features from

omni-directional image A camera can move freely in 3D environment

without knowledge of motion model Algorithm can reconstruct 3D camera path and

3D environment map Test a system in an indoor environment No dynamic objects

Page 31: 3D SLAM for Omni-directional Camera

31

Work Plan Study the works in the related fields Develop algorithms Test the system Evaluate a result Prepare and engage in a thesis defense

Page 32: 3D SLAM for Omni-directional Camera

Thank you

Page 33: 3D SLAM for Omni-directional Camera

33

SLAM Solution Solutions to the SLAM Problem

EKF SLAM - using the extended Kalman filter (EKF) to solve the SLAM problem [Moutarlier and Chatila, 89]

Fast SLAM - using the Rao-Blackwellized particle filter to solve the SLAM problem [Montemerlo et al., 02]

Etc.

Page 34: 3D SLAM for Omni-directional Camera

34

Extended Kalman Filter Kalman Filter

An efficient algorithm for state estimation problems Based on linear dynamical systems A Hidden Markov models with Gaussian distributions

Extended Kalman Filter Nonlinear version of the Kalman filter Using Jacobian to linearize nonlinear functions

Page 35: 3D SLAM for Omni-directional Camera

35

Extended Kalman Filter - Predict

x

P(x)

Initial state

x

bel(x)

Predict

Predicted state

Predicted estimate covariance

state transition function

control vectorprevious state

state transition jacobian previous error covariance

process noise covariance

Page 36: 3D SLAM for Omni-directional Camera

36

Extended Kalman Filter - Update

x

P(x)

Update

Updated state estimate

Updated estimate covariance

predict state

Estimate measurement

Innovation covariance

Optimal Kalman gain

Innovation

innovation

kalman gain

observation function

mesurement

estimate measurement

kalman gain

innovation covariancepredicted estimate covariance

observation jacobian

observation noise covariance

Page 37: 3D SLAM for Omni-directional Camera

37

A state representation represent a camera state and landmarks state

correspond to a motion model and an observation model

Page 38: 3D SLAM for Omni-directional Camera

38

Sensor Sonar Laser range finder Video Camera Omni-directional Camera Etc.

Sonar

Laser range finder Omni-directional CameraVideo Camera

Page 39: 3D SLAM for Omni-directional Camera

39

How SLAM will be used Detect good features from image Find a good state representation for camera

state and landmarks state Find a good observation model from image

features for SLAM Find a good motion model for SLAM

Don’t know camera motion model Need camera motion estimation

Page 40: 3D SLAM for Omni-directional Camera

40

Feature Detection Point Detector

Harris Detector Scale & Affine invariant

point detectors

Line Detector Hough (line) transform

[Ying and Hu, 04]

Page 41: 3D SLAM for Omni-directional Camera

41

Feature Detection Object Detector

Template matching SIFT [Lowe, 04]

Page 42: 3D SLAM for Omni-directional Camera

42

Feature Associations Feature associations from Image

SIFT Optical flow

Feature associations from measurement Mahalanobis distance

Estimate Measurement

Real Measurement

Landmark state

Page 43: 3D SLAM for Omni-directional Camera

43

Camera Motion Estimation Camera can move freely in an environment

Cannot predict camera trajectory Too many state variables to estimate with SLAM only

Need pre-estimate camera state Using relation of two period omni images Non-linear Least Square Iterative Extended kalman filter

Page 44: 3D SLAM for Omni-directional Camera

44

Camera Motion Estimation Known landmark positions related to a reference

frame (from SLAM state) Known current landmark mesurement (from

current image) Can estimate a camera state related to a

reference frame

time k time k + m

camera

landmark

Page 45: 3D SLAM for Omni-directional Camera

45

System Coordinate World Frame Reference Frames Camera Frame

landmark

World Frame

Camera Frame

Reference Frame

Camera

Camera trajectory

Page 46: 3D SLAM for Omni-directional Camera

46

SLAM Prediction

Use a camera estimation state from previous

Measurement Update Use measurements from current and old features

Landmark Augmentation Add a new landmark when a camera encounter a new

feature

Reference Frame Augmentation Select new reference frame from old suitable camera

state

Page 47: 3D SLAM for Omni-directional Camera

47

SLAM Application Indoor Environment

Outdoor Environment[Ahn et al.,07; Davison et al,06; ...]

[Bailey,02; Asmar et al., 06; Miro et al.,06; Newman, 07; Han et al., 07; ...]

Page 48: 3D SLAM for Omni-directional Camera

48

SLAM Application At Home

Under water[Choi et al.,06; Ahn et al,06; Motard et al.,07]

[Williams et al.,01; Ribas et al.,06]

Page 49: 3D SLAM for Omni-directional Camera

49

SLAM Application UAV Navigation

Agricultural Robotics[Jonghyuk Kim et al.,07]

Page 50: 3D SLAM for Omni-directional Camera

50

Type of SLAM Grid maps or scans

Landmark-based

[Lu & Milios, 97; Gutmann, 98: Thrun 98; Burgard, 99; Konolige & Gutmann, 00; Thrun, 00; Arras, 99; Haehnel, 01;...]

[Leonard et al., 98; Castelanos et al., 99: Dissanayake et al., 2001; Montemerlo et al., 2002;...]

Page 51: 3D SLAM for Omni-directional Camera

51

Type of SLAM Topology

Appearance

[Mark Cummins and Paul Newman, 07]

[Motard et al., 07; Valgren et al, 06]

Page 52: 3D SLAM for Omni-directional Camera

52

Problems in SLAM Costs

Computational/Storage, Use of Information form based estimators.

Sensing Issues Interpretation of Sensor Data/Association.

Environmental Representations Feature based, grids, Sum-of-Gaussians, Scan Matching.

Loop Closing Recognising where you have been before.

Scaling Large Environments.

Observability Estimating a high dimensional state with low dimensional

measurements.

Page 53: 3D SLAM for Omni-directional Camera

53

Video Camera Advantage

Information rich Real-time information Compact Inexpensive

Disadvantage Hard to deal with image raw data Not provide depth information Need a lot of computation time

Page 54: 3D SLAM for Omni-directional Camera

54

Our Approach Feature Detection and measurement / Feature

Associations Camera Motion Estimation SLAM