PLANAR VEHICLE TRACKING USING A MONOCULAR BASED MULTIPLE

CAMERA VISUAL POSITION SYSTEM

Anthony HinsonApril 22, 2003

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 2 of 97

Overview

• Introduction• Image Processing

– Primitive– Statistical

• Planar Visual Positioning – Fundamentals– Application

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 3 of 97

Overview

• Testing and Results– Simulation– Actual

• Conclusions• Graphical User Interface• Future Work

– Surface Positioning– Time Based Models

• Demonstration and Questions

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 4 of 97

Link Page

IntroductionPrimitive Image

ProcessingStatistical Image

Processing

TestingPlanar Positioning

FundamentalsPlanar Positioning

Application

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 5 of 97

Link Page

Conclusions Future WorkGraphical User

Interface

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 6 of 97

Introduction

• Simple Monocular Vision Based Position System for Tracking of Indoor and Outdoor Vehicles

Concept

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 7 of 97

Introduction

• Uses Single or Multiple Cameras to Determine Vehicle Position and Orientation

Concept

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 8 of 97

Introduction

• Vehicle Position and Orientation Determined Via Tracking Features On Top of the Vehicle

Concept

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 9 of 97

Introduction

• Advantages– Well Suited for Indoor Vehicles– Accurate Position Information– Easy to Implement– Non-Intrusive to Environment or Vehicle– Not Specific to Certain Hardware– One-Time Setup

Concept

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 10 of 97

Introduction

• Advantages– Video Feed Can

Be Used for Monitoring and Positioning Simultaneously

Concept

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 11 of 97

Introduction

• Disadvantages– Reliability is Dependent on Environmental

Conditions– Accuracy Decreases with Range– Planar Positioning System (2D Only)

Concept

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 12 of 97

Image Processing

• Initial Image Processing Work– Some Routines Good for Basic Image

Enhancement– Largely Ineffective for Feature Extraction

Primitive

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 13 of 97

Image Processing

• ColorBias– Process – Shifts Individual Color Channel Values– Usage – Used for Hue Correction – Synopsis – Reasonably Fast and Effective

Primitive

Modified ImageOriginal Image

ColorBias

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 14 of 97

Image Processing

• ProgressiveSmooth– Process – Performs Weighted Averaging with

Neighboring Pixels– Usage – Used for Noise Removal and Anti-Aliasing– Synopsis – Effective but Slow

Primitive

Modified ImageOriginal Image

ProgressiveSmooth

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 15 of 97

Image Processing

• ColorDistinguish– Process – Removes Pixels that Are not Within the

User-Specified Range – Usage – Color Feature Extraction– Synopsis – Limited Functionality / No Longer Used

Primitive

Modified ImageOriginal Image

ColorDistinguish

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 16 of 97

Image Processing

• ColorRemove– Process – Removes Pixels that Are not Within the

User-Specified Range – Usage – Removes Unwanted Colors– Synopsis – Limited Functionality / No Longer Used

Primitive

Modified ImageOriginal Image

ColorRemove

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 17 of 97

Image Processing

• Threshold– Process – Removes Pixels with Values Less Than User-

Specified Boundary– Usage – Removes Dark Pixels / Was Typically Used to

Enhance Edge Information– Synopsis – No Longer Used

Primitive

Modified ImageOriginal Image

Threshold

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 18 of 97

Image Processing

• EdgeDetect– Process – Calculates Color Discrepancy Between

Adjacent Pixels – Usage – Finds Edges of Color Boundaries– Synopsis – Relatively Fast and Effective

Primitive

Modified ImageOriginal Image

EdgeDetect

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 19 of 97

Image Processing

• ScreenText– Process – Writes Alphanumeric Characters to a Video

Pixel Array– Usage – Currently Used to Display Range Data in

Video Stream– Synopsis – Works Very Well

Primitive

Modified ImageOriginal Image

ScreenText

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 20 of 97

Image Processing

• Primitive Image Processing Functions Insufficient for Visual Positioning – Work Reasonably Well on Simulated Images– Work Poorly on Experimental Images

Primitive

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 21 of 97

Image Processing

• Desired Capabilities of Feature Classifier– Capable of Handling Simulated Data– Capable of Handling Experimental Data– Fast Processing Speed

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 22 of 97

Image Processing

• Color Space (RGB Space)– All Possible

Digital Colors Represented by Cube with Dimension of 256

– Each Axis Represents Color

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 23 of 97

Image Processing

• In RGB Space– Color Distributions Have Physical Meaning– Distributions Can be Represented by 3D

Shapes in RGB Space

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 24 of 97

Image Processing

• In RGB Space– Data from an

Image Can be Displayed as Data Points in RGB Space

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 25 of 97

Image Processing

• Color Classifiers Used in This Research– Color Range– Normalized Color Direction– 3D Gaussian Color Distribution– 2D Normalized Gaussian Color Distribution

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 26 of 97

Image Processing

• Color Range– Basically Same as

ColorDistinguish – Distribution Defined

by High & Low Values for Each Color Channel Separately

– Distribution is Represented by a Box in RGB Space

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 27 of 97

Image Processing

• Color Range– High/Low Values Determined

By 1-D Gaussian Distributions for Each Color Channel

• High Value = + n• Low Value = – n

– Pixels Located Inside the Box are Considered to be Target Color

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 28 of 97

Image Processing

• Color Range– Advantages

• Very Fast

– Disadvantages• Not Very Precise• Typically Yields High Error

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 29 of 97

Image Processing

Color Range Sample Image

Statistical

Processed ImageOriginal Image

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 30 of 97

Image Processing

• Color Range in RGB Space– Black: Correctly

Classified Non-Feature pixels

– White: Correctly Classified Feature Pixels

– Blue: Missed Feature Pixels

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 31 of 97

Image Processing

• Color Direction– Searches for Pixels

Using Color Vectors in RGB Space

– Distribution is Defined as a Target Color and Range

– Resulting Distribution Shape is a Conic Section

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 32 of 97

Image Processing

• Color Direction– Color Normalization Equations

• Converts Discreet Color Value to Normalized Color Direction Vector

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 33 of 97

Image Processing

• Color Direction– Distribution Defined By:

• Target Color (Mean of Normalized Feature Pixels)

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 34 of 97

Image Processing

• Color Direction– Distribution Defined By:

• Color Direction Variance (Each Color Separate)

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 35 of 97

Image Processing

• Color Direction– Distribution Defined By:

• Any Pixel with a Color Direction Between + nand – nis Considered to be Feature Pixel

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 36 of 97

Image Processing

• Color Direction– Advantages

• Discards Brightness Information• Can Find Colors in the Light or Shadows• Inherently Compensates for Scattered Color Data

– Disadvantages• More Likely to Have False Hits on Similar Colored

Objects in Scene

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 37 of 97

Image Processing

Color Direction Sample Image

Statistical

Processed ImageOriginal Image

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 38 of 97

Image Processing

• Color Direction RGB Space – Black: Correctly

Classified Non-Feature pixels

– White: Correctly Classified Feature Pixels

– Blue: Missed Feature Pixels

– Red: False Hit Pixels

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 39 of 97

Image Processing

• 3D Gaussian Distribution– Classifies Data

According to a Normal Distribution

– Classifier is Represented by a 3D Ellipsoid in RGB Space

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 40 of 97

Image Processing

• 3D Gaussian Distribution– Classifier’s Shape and Position are Defined

By:• Mean Color of Feature Data• Variance Within Each Color Channel• Covariance Between Color Channel

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 41 of 97

Image Processing

• 3D Gaussian Distribution– Probability Density Function (PDF)

– Where

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 42 of 97

Image Processing

• 3D Gaussian Distribution– Variance Calculations

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 43 of 97

Image Processing

• 3D Gaussian Distribution– Exponential Part of PDF Can be Used to

Assess Membership of Pixel to the Distribution

– r is known as Mahalanobis Distance

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 44 of 97

Image Processing

• 3D Gaussian Distribution– Mahalanobis Distance

• The Number of Standard Deviations The Current Pixel is from the Mean

• Any Pixel with an r of Less Than User-Specified Value is Considered Member of Distribution

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 45 of 97

Image Processing

• 3D Gaussian Distribution– Advantages

• Very Accurate for Most Distributions• Compensates for Data Clusters at Any Location

and Orientation in RGB Space

– Disadvantages• Color Distribution Must Be Relatively Gaussian in

Distribution

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 46 of 97

Image Processing

3D Gaussian Distribution Sample Image

Statistical

Processed ImageOriginal Image

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 47 of 97

Image Processing

• 3D Gaussian RGB Space – Black: Correctly

Classified Non-Feature pixels

– White: Correctly Classified Feature Pixels

– Blue: Missed Feature Pixels

– Red: False Hit Pixels

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 48 of 97

Image Processing

• 2D Normalized Gaussian Distribution– Hybrid of 3D

Gaussian and Color Direction

– Converts 3D Color Cube to 2D Color Triangle

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 49 of 97

Image Processing

• 2D Normalized Gaussian Distribution– Color Data Reduced to 2 Dimensions

• Removes Brightness Information• Bivariate Gaussian Classifier

– Classifier Shape is an Ellipse within the Color Triangle

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 50 of 97

Image Processing

• 2D Normalized Gaussian Distribution– Color Data Flattening (Convert RGB

Coordinates to XY Coordinates)

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 51 of 97

Image Processing

• 2D Normalized Gaussian Distribution– Multivariate Distribution

– Where

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 52 of 97

Image Processing

• 2D Normalized Gaussian Distribution– Advantages

• Same As Color Direction Classifier • Allows for Better Classification Than Color

Direction

– Disadvantages• Same As Color Direction Classifier • Slower Than Color Direction

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 53 of 97

Image Processing

2D Normalized Gaussian Distribution Sample Image

Statistical

Processed ImageOriginal Image

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 54 of 97

Image Processing

• RGB Space 2D Normalized Gaussian Distribution– Black: Correctly

Classified Non-Feature pixels

– White: Correctly Classified Feature Pixels

– Blue: Missed Feature Pixels

– Red: False Hit Pixels

Statistical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 55 of 97

Planar Positioning

• Planar Positioning Concepts– Camera View Compresses 3D View to 2D– Each Pixel Represents a Vector to an Object

in Space– Distance to the Object is Unknown– Point at Where Pixel Vector Intersects Object

in Space Must be Found

Concepts

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 56 of 97

Planar Positioning

• Planar Positioning Concepts– Intersection Can be Found if Pixel Vector

Intersects a Plane– Each Pixel Will Represent a Finite Area on

the Plane

Concepts

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 57 of 97

Planar Positioning

• Quantities Needed For Reconstruction of 3D Data– Extrinsic Camera Properties

• X, Y, and Z Coordinates of Camera• Pan, Tilt, and Slant Angles of Camera

– Intrinsic Camera Properties• Field of View in Horizontal and Vertical

– Video Capture Device Properties• Resolution of Video Capture

– Planar Properties• Coordinates of Plane (D;A,B,C)

Concepts

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 58 of 97

Planar Positioning

• Determining Required Input Data– Tracking Plane Must be Defined (Typically Parallel or

Coincident with Ground)– Camera Must be Placed in Position to Be Able to See

Plane– Video Capture Hardware Must be Initialized to

Determine Capture Resolution

Procedure

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 59 of 97

Planar Positioning

Creating Tracking Data Lookup Table (LUT)

Procedure

• Define Pixel Grid– Camera Placed at

Home Location– Image Plane

Assumed to be at Unit Distance from Origin in Y-Direction

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 60 of 97

Planar Positioning

Creating Tracking Data Lookup Table (LUT)

Procedure

• Define Pixel Grid– Image Plane

Boundaries Determined Trigonometrically Using Fields of View in Horizontal and Vertical

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 61 of 97

Planar Positioning

Creating Tracking Data Lookup Table (LUT)

Procedure

• Define Pixel Grid– Image Plane Area

Divided Into Pixel Grid Corresponding to Capture Resolution

– Intersections of Gridlines are Referred to as Pixel Grid Nodes

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 62 of 97

Planar Positioning

Creating Tracking Data Lookup Table (LUT)

Procedure

• Define Pixel Grid– Pixel Grid Node

Locations are Recorded in Homogenous Coordinates Format

(w;x,y,z)

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 63 of 97

Planar Positioning

• Translate & Rotate Pixel Grid – Pixel Grid Points

Translated to Camera XYZ Location By Multiplying Each Point by Translation Matrix

Procedure

Creating Tracking Data Lookup Table (LUT)

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 64 of 97

Planar Positioning

• Translate & Rotate Pixel Grid – Pixel Grid Points

Rotated to Camera Orientation By Multiplying Each Point by Three Rotation Matrices

Procedure

Creating Tracking Data Lookup Table (LUT)

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 65 of 97

• Translation and Rotation Matrices

Planar PositioningProcedure

Creating Tracking Data Lookup Table (LUT)

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 66 of 97

Planar Positioning

• Create Pixel Node Vectors – Vectors Created

Between Focal Point of Camera and Pixel Grid Nodes

Procedure

Creating Tracking Data Lookup Table (LUT)

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 67 of 97

Planar Positioning

• Create Pixel Node Vectors – Vectors Represented

in Terms of Plücker Line Coordinates

or

Procedure

Creating Tracking Data Lookup Table (LUT)

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 68 of 97

Planar Positioning

• Create Planar Intersection Points– Intersection Between

All Vectors and Plane Can be Found

Procedure

Creating Tracking Data Lookup Table (LUT)

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 69 of 97

Planar Positioning

• Projective Geometry – Intersection of Line and Plane Determine a Point

Equation of Line Equation of Plane

Intersection of Line and Plane=

Procedure

Creating Tracking Data Lookup Table (LUT)

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 70 of 97

Planar Positioning

• Determine Pixel Areas– Each Pixel Node

Intersection Point Corresponds to the Corner of a Pixel Area

Procedure

Creating Tracking Data Lookup Table (LUT)

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 71 of 97

Planar Positioning

• Calculate Pixel Centroids– Pixel Centroid is the

Average of the Four Corners of Pixel Area

– The Centroid Represents the Coordinates that the Pixel Represents in Space

Procedure

Creating Tracking Data Lookup Table (LUT)

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 72 of 97

Planar Positioning

• Calculate Pixel Centroids– Error Represented by

Maximum Distance from Centroid to Area Vertex

Procedure

Creating Tracking Data Lookup Table (LUT)

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 73 of 97

Planar Positioning

• Environment Setup for Planar Positioning– Vehicle Drive Path Must be Planar– Cameras Must Cover All Drive Areas

Application

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 74 of 97

Planar Positioning

• Vehicle Setup for Planar Positioning– Vehicle Must Have 2 Tracking Features

Distinguishable from Rest of Image Residing in a Plane Parallel to Ground

Application

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 75 of 97

Planar Positioning

• Setup for Planar Positioning– Camera Properties Must be Precisely Defined

• Intrinsic• Extrinsic

– Environment Must be Accurately Mapped• Boundaries • Obstacles

– Tracking Plane Must be Defined as the Plane the Tracking Features are in

Application

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 76 of 97

Planar Positioning

• Using Planar Positioning– Tracking

Information is Displayed for Allowed Areas

Application

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 77 of 97

Testing & Results

• Test #1: Simulated Warehouse

Test #1

– Three Camera Views• Camera2• Camera3• Camera6

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 78 of 97

Testing & ResultsTest #1

• Initial Test– Gridline

Match up• Check to See

if Grid Lines Up With Walls

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 79 of 97

Testing & Results

• Test #1: Simulated Warehouse

Test #1

– Initial Test• Gridline

Match up

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 80 of 97

Testing & Results

• Results– Red:

Measured– Green:

Camera2– Blue:

Camera3– Magenta:

Camera6

Test #1

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 81 of 97

Testing & ResultsTest #1

• Results– Error Typically Less Than 1%– Some Feature Classifier Break-Down at Far Distances

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 82 of 97

Testing & Results

• Test #2: Desktop Rover

Test #2

– Miniature Remote Control Tank-like Vehicle

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 83 of 97

Testing & ResultsTest #2

• Test Setup– 2 Cameras– Poster board

grid • 4x4 Major

Gridlines• 1x1 Minor

Gridlines

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 84 of 97

Testing & ResultsTest #2

• Initial Gridline Test– Software Gridlines

Overlay Match Existing Gridlines Well

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 85 of 97

Testing & ResultsTest #2

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 86 of 97

Testing & ResultsTest #2

• Results– Blue:

Camcorder– Magenta:

Sony CCD– Yellow:

Calculated Position

– Red: Measured Position

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 87 of 97

Testing & ResultsTest #2

• Results– 1% to 2% Error Typically– Slightly Higher Error from Sony CCD Camera at More

Distant Locations– Vehicle Location Lost Occasionally from Camcorder

Video

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 88 of 97

Testing & ResultsTest #3

• Test #3: Remote Controlled Truck– Inexpensive

Radio Controlled Truck

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 89 of 97

Testing & ResultsTest #3

• Test Setup– 3 Camera Test

• Panasonic Camcorder• Sony XC711 Industrial Camera• X10 Wireless Camera (Onboard)

– Vehicle Tested on Tiled Floor Space• 8x8 Inch Floor Tiles Used as External Reference

Point for Analyzing Tracking Data

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 90 of 97

Testing & ResultsTest #3

• Initial Gridline Test– Gridlines

Match Tile Grid Well

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 91 of 97

Testing & ResultsTest #2

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 92 of 97

Testing & ResultsTest #3

• Results– Blue:

Camcorder– Magenta:

Sony CCD– Yellow:

Calculate Position

– Red: Measured Position

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 93 of 97

Testing & ResultsTest #3

• Results– Significant Classifier Breakdown with Distance or

Lighting Changes– Problematic Camera Model for Sony CCD Camera– Data from Sony CCD Camera Stayed Within 3% Error

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 94 of 97

Testing & ResultsTest #4

• Test #4: Warehouse Test– Lighting

Conditions Very Poor

– Feature Color Information Washed Out

– Test had to be Discarded

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 95 of 97

Conclusions

• Planar Visual Position System Works Well When:– Vehicle and Environment are Measured Well– Camera Properties are Known – Classifiers are Well-Defined

• Classification Technique Needs to Be Improved – Works Well with Simulations and Controlled

Environments– Classifier Breaks Down when Conditions Become Bad

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 96 of 97

Future Work

• Adapt Live Video Capabilities• Surface Positioning

– Extend to Non-Planar Surfaces

Cen

ter

for

Inte

llig

en

t M

ach

ines a

nd

Rob

oti

cs

Slide 97 of 97

Questions & Demo

Questions ?