GCAPS TeamGCAPS TeamDesign ReviewDesign Review

CPE 450 Section 1CPE 450 Section 1January 22, 2008January 22, 2008

Nick HebnerNick HebnerKooper FrahmKooper Frahm

Ryan WeissRyan Weiss

IntroductionIntroduction►General Problem StatementGeneral Problem Statement

In order to create an autonomous In order to create an autonomous transportation system, both position and transportation system, both position and orientation information is required. This orientation information is required. This project entails the development of a project entails the development of a reliable position data acquisition system reliable position data acquisition system and a means of translating it into useful and a means of translating it into useful state information. The vehicle must self-state information. The vehicle must self-localize based on its knowledge of localize based on its knowledge of surroundings.surroundings.

IntroductionIntroduction►Client - Chris ClarkClient - Chris Clark

Cal Poly professorCal Poly professor Specialized experience with Specialized experience with

Artificial Intelligence, Artificial Intelligence, Autonomous Mobile Robots, Autonomous Mobile Robots, and Multi-Robot Systemsand Multi-Robot Systems

Founded the Cal Poly Founded the Cal Poly Autonomous Transportation Autonomous Transportation System Project (CPATS)System Project (CPATS)

IntroductionIntroduction►Related WorkRelated Work

Precise Vehicle Localization Using Multiple Precise Vehicle Localization Using Multiple Sensors and Natural LandmarksSensors and Natural Landmarks

►GPS, GIS Mapping, Visual Landmarks, Kalman GPS, GIS Mapping, Visual Landmarks, Kalman FilterFilter

A Visual Positioning System for Vehicle or A Visual Positioning System for Vehicle or Mobile Robot NavigationMobile Robot Navigation

►Stereo Vision, Feature extraction/trackingStereo Vision, Feature extraction/tracking 2D Map-Building and Localization in 2D Map-Building and Localization in

Outdoor EnvironmentsOutdoor Environments►Laser Landmark Detection, Kalman Filter, GPSLaser Landmark Detection, Kalman Filter, GPS

Formal Product DefinitionFormal Product Definition► Need StatementNeed Statement

Current vehicles are often prone to failure when Current vehicles are often prone to failure when mistakes are made by the operators. Of the many mistakes are made by the operators. Of the many possible systems that could aid operators, one of possible systems that could aid operators, one of the most basic and important is a self-localization the most basic and important is a self-localization system which models the state of the vehicle at all system which models the state of the vehicle at all times.times.

Ultimately, transportation systems may be Ultimately, transportation systems may be operated entirely by computer systems, providing operated entirely by computer systems, providing a safer and more efficient ride. Among the a safer and more efficient ride. Among the requirements for such a system is the ability to requirements for such a system is the ability to know the current position and use this information know the current position and use this information to efficiently transport people.to efficiently transport people.

Formal Product DefinitionFormal Product Definition► ObjectivesObjectives

Self Localization.Self Localization. The system needs to know where The system needs to know where it is at all times with relation to its surroundings. it is at all times with relation to its surroundings.

Generation of a meaningful sensor and visual Generation of a meaningful sensor and visual image processing algorithm.image processing algorithm. The system will The system will implement a mechanism for making decisions and implement a mechanism for making decisions and signaling if any changes are necessary. The signaling if any changes are necessary. The algorithm will need to process data received from algorithm will need to process data received from the image processing unit in conjunction with the the image processing unit in conjunction with the data from the wheel encoder and translate this data from the wheel encoder and translate this into meaningful action.into meaningful action.

State Estimation.State Estimation. The system needs to track its The system needs to track its position (x,y) and heading (θ) around a small loop position (x,y) and heading (θ) around a small loop providing live feedback. providing live feedback.

Formal Product DefinitionFormal Product Definition►RequirementsRequirements

MarketingMarketing►The system should be able to distinguish unique The system should be able to distinguish unique

markings to help with self-location.markings to help with self-location.►The system should be able to determine The system should be able to determine

distance traveled between any given markings, distance traveled between any given markings, and total distance traveled.and total distance traveled.

►The system should be fully autonomous, The system should be fully autonomous, meaning completely run from a machine in meaning completely run from a machine in conjunction with a computer, without any conjunction with a computer, without any human interaction.human interaction.

Formal Product DefinitionFormal Product Definition►RequirementsRequirements

EngineeringEngineering►The system will use a Particle Filter (fusion The system will use a Particle Filter (fusion

system) to make positional determinations system) to make positional determinations based on image processing and wheel encoding.based on image processing and wheel encoding.

►The system must support image detection as a The system must support image detection as a primary means of determining position.primary means of determining position.

►The system must support wheel encoding to The system must support wheel encoding to provide a distance and vector change to all provide a distance and vector change to all possible “particles” generated by the Particle possible “particles” generated by the Particle Filter.Filter.

Formal Product DefinitionFormal Product Definition►RequirementsRequirements

Engineering (cont.)Engineering (cont.)►The system will have an onboard computer The system will have an onboard computer

that controls all functions of golf-cart that controls all functions of golf-cart operation.operation.

►The system will have an onboard camera that The system will have an onboard camera that has auto focus.has auto focus.

►The system will have wheel encoders, both The system will have wheel encoders, both incremental and absolute to provide distance incremental and absolute to provide distance and angle measurements, respectively.and angle measurements, respectively.

Formal Product DefinitionFormal Product Definition► RequirementsRequirements

ConstraintsConstraints► A vision system is to be used to detect visual A vision system is to be used to detect visual

landmarks that are used by the fusion system in landmarks that are used by the fusion system in determining position.determining position.

► A second sensor system (encoding) needs to provide A second sensor system (encoding) needs to provide distance and angle measurements for all particles in the distance and angle measurements for all particles in the system.system.

► Position must be known within an accuracy of 10 cm.Position must be known within an accuracy of 10 cm.► The system must navigate and determine its position The system must navigate and determine its position

while moving around a circle.while moving around a circle.

Formal Product DefinitionFormal Product Definition►CriteriaCriteria

Positional detection error (smaller = better) Positional detection error (smaller = better) Frequency of positional updates (higher = Frequency of positional updates (higher =

better) better) Computational time (lower = better)Computational time (lower = better) Artificial landmark dimensions (smaller = Artificial landmark dimensions (smaller =

better) better) Deployment cost (time) (smaller = better) Deployment cost (time) (smaller = better) Deployment cost (money) (smaller = better)Deployment cost (money) (smaller = better) Production cost (smaller = better)Production cost (smaller = better)

DesignDesign►Overview of Team ProcessOverview of Team Process

Problem StatementProblem Statement Requirements SpecificationRequirements Specification DesignDesign ImplementationImplementation TestingTesting MaintenanceMaintenance Final ReportFinal Report

DesignDesign► System ArchitectureSystem Architecture

Particle Filter

Dead Reckoning

Position Transformation

(pose estimation)

Landmark Detection

(PCA/KLT)

(Xv0,Yv0,Θv0) (image coordinates)

Camera

Image

(Xv,Yv,Θv) estimate(world coordinates)

(Xe,Ye,Θe) estimate(world coordinates)

Odometry Encoder

Wheel Angle

Encoder

Travel Distance Delta Wheel Angle

Software

Hardware

Estimated System State (X,Y,Θ)

(world coordinates)

DesignDesign► Particle FilterParticle Filter

The fusion system that combines data from the The fusion system that combines data from the image and encoding systems to determine image and encoding systems to determine location.location.

► OdometryOdometry The system that measures distance and angle of The system that measures distance and angle of

the cart by dead reckoning and passes the data the cart by dead reckoning and passes the data to the fusion algorithm.to the fusion algorithm.

Dead Reckoning simply estimates one’s current Dead Reckoning simply estimates one’s current position based upon knowledge of a previous position based upon knowledge of a previous position advanced by a heading and distance.position advanced by a heading and distance.

DesignDesign► Image ProcessingImage Processing

The system that takes images and uses a The system that takes images and uses a technique called Principle Components technique called Principle Components Analysis (PCA) to determine position and Analysis (PCA) to determine position and send the data to the fusion system.send the data to the fusion system.

Pose of the landmark is extracted by its Pose of the landmark is extracted by its similarity to known poses of landmarks in similarity to known poses of landmarks in the training images.the training images.

DesignDesign►Hardware DesignHardware Design

The camera is interfaced to the computer The camera is interfaced to the computer by USB. Images are captured from the by USB. Images are captured from the camera using OpenCV.camera using OpenCV.

The encoder is interfaced to a The encoder is interfaced to a microcontroller which constantly polls for microcontroller which constantly polls for changes. Updates are sent at intervals to changes. Updates are sent at intervals to the host.the host.

DesignDesign►Software AlgorithmsSoftware Algorithms

PCA – used for determining PCA – used for determining position from pose estimation. position from pose estimation.

►Available through OpenCV and/or Available through OpenCV and/or Matlab.Matlab.

Dead ReckoningDead Reckoning►The algorithm uses distance traveled and The algorithm uses distance traveled and

wheel angle to calculate changes in position.wheel angle to calculate changes in position.►Wheel angle -> Turning RadiusWheel angle -> Turning Radius

DesignDesign► Software Algorithms (cont.)Software Algorithms (cont.)

Particle Filtering (Loop over time step t)Particle Filtering (Loop over time step t)►For i = 1 …N►Pick xt-1[i] from Xt-1►Draw xt[i] with probability p( xt[i]| xt-1[i], ot)►Calculate wt[i]= p( zt| xt[i])►Add xt[i] to XtTemp►For j = 1 …N►Draw xt[j] from XtTemp with probability wt[j]►Add xt[i] to Xt► Steps 1 through 5 represent the Prediction steps, 6 Steps 1 through 5 represent the Prediction steps, 6

through 8 are the correction steps.through 8 are the correction steps.

DesignDesign►Mechanical DesignMechanical Design

Wheel encoders must be fastened to the Wheel encoders must be fastened to the test vehicle for appropriate measurement.test vehicle for appropriate measurement.

The mechanical design still needs thought The mechanical design still needs thought as the parts have not arrived and method as the parts have not arrived and method of fastening the encoders and the camera of fastening the encoders and the camera is not clear.is not clear.

DesignDesign► ComponentsComponents

Image Processing - tImage Processing - the responsibility of the image he responsibility of the image processing component is to receive data and process the processing component is to receive data and process the information. Small markings will be used as landmarks for information. Small markings will be used as landmarks for the system to locate itself and ensure proper operation. the system to locate itself and ensure proper operation.

Landmarks - tLandmarks - the landmarks will be small markings on the he landmarks will be small markings on the faces of curbs or the street that the system uses to faces of curbs or the street that the system uses to determine its location. determine its location.

Wheel Encoder - tWheel Encoder - the responsibility of the wheel encoder is he responsibility of the wheel encoder is to provide a mechanical means of data to be passed to the to provide a mechanical means of data to be passed to the Particle Filter. The wheel encoder is able to count the Particle Filter. The wheel encoder is able to count the rotations of the wheel and accurately pass this information rotations of the wheel and accurately pass this information to the fusion algorithm. The encoder will be mounted in to the fusion algorithm. The encoder will be mounted in one of the wheel hubs to enable accurate counting.one of the wheel hubs to enable accurate counting.

DesignDesign►Components (cont.)Components (cont.)

Laptop PC - tLaptop PC - the responsibility of the laptop he responsibility of the laptop is to provide the processing capability of all is to provide the processing capability of all the necessary components. The platform the necessary components. The platform will be an x86 architecture running a Linux will be an x86 architecture running a Linux operating system.operating system.

Processing Algorithm - tProcessing Algorithm - the main processing he main processing of the system will be accomplished with of the system will be accomplished with the use of a Particle Filter. the use of a Particle Filter.

Test PlanTest Plan► PCA TestPCA Test

Synthesized imagesSynthesized images Input live image, compare to correlated training Input live image, compare to correlated training

imageimage► Straight Line TestStraight Line Test

Initialize Golf-cart’s positionInitialize Golf-cart’s position Flat, feature-absent roadFlat, feature-absent road Mid-day, typical lightingMid-day, typical lighting No GUI, measure and compareNo GUI, measure and compare

► Feature-rich RoadsFeature-rich Roads Road may contain natural features and deformationsRoad may contain natural features and deformations

► Curved Paths & Circular PathsCurved Paths & Circular Paths Realistic roadsRealistic roads U-turnsU-turns

Budget and JustificationBudget and Justification►The budget has not been determined, The budget has not been determined,

and therefore cannot be justified.and therefore cannot be justified.

Bill of MaterialsBill of Materials►Logitech QuickCam Pro (for notebooks) Logitech QuickCam Pro (for notebooks)

$57.06$57.06►Quadrature Wheel Encoder Modules Quadrature Wheel Encoder Modules

$57.94$57.94►Total Total

$115.00$115.00

ReferencesReferences► Precise Vehicle Localization Using Multiple Sensors Precise Vehicle Localization Using Multiple Sensors

and Natural Landmarksand Natural Landmarks Ullrich Scheunert, Heiko Cramer, Gerd WanielikUllrich Scheunert, Heiko Cramer, Gerd Wanielik

► A Visual Positioning System for Vehicle or Mobile A Visual Positioning System for Vehicle or Mobile Robot NavigationRobot Navigation Huei-Yung Lin, Jen-Hung LinHuei-Yung Lin, Jen-Hung Lin

► 2D Map-Building and Localization in Outdoor 2D Map-Building and Localization in Outdoor EnvironmentsEnvironments R. Madhavan, H. F. Durrant-WhyteR. Madhavan, H. F. Durrant-Whyte

► Visual Learning and Recognition of 3-D Objects Visual Learning and Recognition of 3-D Objects from Appearancefrom Appearance H. Murase, S. K. NayarH. Murase, S. K. Nayar

Gantt ChartGantt Chart