Upload
cchoi
View
1.277
Download
0
Tags:
Embed Size (px)
DESCRIPTION
This slides were for presentation in IROS 2008 conference.
Citation preview
Real-time 3D Object Pose Estimation and Tracking for Natural Landmark Based Visual Servo
Seung-Min Baek and Sukhan LeeSungkyunkwan University
Intelligent System Research Center
Changhyun ChoiGeorgia Tech
College of Computing
Contents• Introduction– Motivation– Related Works
• Proposed Approach– System Overview– Problem Definition– Initial Pose Estimation– Local Pose Estimation
• Experimental Results• Summary & Conclusion• Future Work
IEEE/RSJ IROS 2008, Sept 25 2
Introduction
In Visual Servo Control,• Object Recognition • Pose Estimation are key tasks.
IEEE/RSJ IROS 2008, Sept 25 3
Introduction
Many systems still useArtificial Landmark.
Unnatural in human environment
IEEE/RSJ IROS 2008, Sept 25 4
Introduction
We need Natural Landmarks.
Natural Landmarks are visual features objects inherently have.IEEE/RSJ IROS 2008, Sept 25 5
Introduction
Modern recognition methods
SIFTabout 200~300 ms on a modern PC
Structured lightseveral seconds
IEEE/RSJ IROS 2008, Sept 25 6
Motivation
• How to apply these state-of-the-art recognition methods to visual servo control?
• How to overcome the time lag?
• How to solve the real-time issue?
IEEE/RSJ IROS 2008, Sept 25 7
Related Works
• Monocular • Model-based• Use keyframe information
as prior knowledge• Use sparse bundle
adjustment technique[ L. Vacchetti et al., PAMI 04 ]
Input image should be close enough to the prior knowledge!
IEEE/RSJ IROS 2008, Sept 25 8
Related Works
• Active Contour• Local curve fitting
algorithm• Initialize by SIFT keypoint
matching
[G. Panin and A. Knoll, JMM 04 ]
Potential danger in background having same color with tracking object!
IEEE/RSJ IROS 2008, Sept 25 9
Our Idea
• Use prior knowledge (object models)– 2D images– 3D points obtained from structured light system
• Use scale invariant feature matching for accurate initialization
• Use KLT (Kanade-Lucas-Tomasi) tracker for fast local tracking
IEEE/RSJ IROS 2008, Sept 25 10
System Overview
• Add text
IEEE/RSJ IROS 2008, Sept 25 11
Two Modes
• Mono Mode– Using mono camera– Better computational performance
• Stereo Mode– Using stereo camera– More accurate pose result
IEEE/RSJ IROS 2008, Sept 25 12
Problem Definition – Mono Mode
Given 2D-3D correspondences and a calibrated mono camera, find the pose of the object with
respect to the camera.IEEE/RSJ IROS 2008, Sept 25 13
Problem Definition – Stereo Mode
Given 3D-3D correspondences and a calibrated stereo camera, find the pose of the object
with respect to the camera.IEEE/RSJ IROS 2008, Sept 25 14
Initial Pose Estimation
• Add text
IEEE/RSJ IROS 2008, Sept 25 15
Initial Pose Estimation
1. Extract SIFT keypoints2. Matching with model
knowledge3. Estimate initial pose4. Get a convex hull of a set of
matched SIFT keypoints5. Generate KLT tracking points
within the convexhull6. Calculate 3D coordinates of
KLT points
IEEE/RSJ IROS 2008, Sept 25 16
Initial Pose Estimation
• Mono Mode– Use the POSIT algorithm (2D-
3D)
• Stereo Mode– Use the closed-form solution
using unit quaternions (3D-3D)
R,tR,t
R,tR,t
IEEE/RSJ IROS 2008, Sept 25 17
Initial Pose Estimation
1. Extract SIFT keypoints2. Matching with model
knowledge3. Estimate initial pose4. Get a convex hull of a set of
matched SIFT keypoints5. Generate KLT tracking points
within the convexhull6. Calculate 3D coordinates of
KLT points
IEEE/RSJ IROS 2008, Sept 25 18
Initial Pose Estimation• 3D coordinates of each KLT points are required for
subsequent local pose estimation
• Stereo Mode– Straightforward in a calibrated stereo rig– Triangulate 3D points
• Mono Mode– Use approximation with the knowledge of model– Get 3D coordinates by using three nearest neighboring
SIFT points
IEEE/RSJ IROS 2008, Sept 25 19
Initial Pose Estimation
+ : SIFT points• : KLT points
IEEE/RSJ IROS 2008, Sept 25 20
Initial Pose Estimation
Treat the surface as locally flatIEEE/RSJ IROS 2008, Sept 25 21
Local Pose Estimation
• Add text
IEEE/RSJ IROS 2008, Sept 25 22
Local Pose Estimation
• Estimate pose with KLT tracking points and their 3D points
• Pose estimation algorithms are same– Mono Mode
• Use the POSIT algorithm (2D-3D)
– Stereo Mode• Use the closed-form solution using unit
quaternions (3D-3D)
R,tR,t
R,tR,t
IEEE/RSJ IROS 2008, Sept 25 23
Removing Outliers
IEEE/RSJ IROS 2008, Sept 25 24
Outlier Handling• KLT tracking points are easy to
drift
• Drifting points result in inaccurate pose
• Use RANSAC to remove outlier
• Re-initialize when there are no sufficient # of inliers
IEEE/RSJ IROS 2008, Sept 25 25
Tracking Results
IEEE/RSJ IROS 2008, Sept 25 26
Experiment
Mono Mode
Stereo ModeIEEE/RSJ IROS 2008, Sept 25 27
Tracking Results - translation
IEEE/RSJ IROS 2008, Sept 25 28
Tracking Results - rotation
IEEE/RSJ IROS 2008, Sept 25 29
RMS Error
RMS errors over the whole sequence of image
Z
IEEE/RSJ IROS 2008, Sept 25 30
Computational Time
Computational times of pose estimationIEEE/RSJ IROS 2008, Sept 25 31
Computational Time
Computational times of each module
IEEE/RSJ IROS 2008, Sept 25 32
Summary & Conclusion
• A method for tracking 3D roto-translation of rigid objects – using scale invariant feature based matching – KLT (Kanade-Lucas-Tomasi) tracker
• Mono mode– guarantees higher frame rate performance
• stereo mode– shows better pose results
IEEE/RSJ IROS 2008, Sept 25 33
Future Work
• To decrease the computational burden– Use GPU-based implementation of KLT tracker and
SIFT• GPU KLT• SiftGPU
– Unifying the contour based tracking
IEEE/RSJ IROS 2008, Sept 25 34
Thank you
• Any Questions?
• Any Suggestions?
• Any Comments?
IEEE/RSJ IROS 2008, Sept 25 35