Upload
vokhuong
View
213
Download
0
Embed Size (px)
Citation preview
Metaheuristics and Cognitive Models for Autonomous
Robot Navigation
Raj Korpan
Department of Computer Science
The Graduate Center, CUNY
Second Exam Presentation
April 25, 2017
1 / 31
Autonomous robot navigation (ARN)
• Mobile robots move through an environment from one location to another without human intervention
• Two heuristic-based approaches
• Metaheuristics for path planning
• Cognitive models of human navigation
• These approaches
• Learn about the search space or the environment
• Balance the tradeoff between exploration and exploitation
2 / 31
Outline
• Background
• Challenges in ARN
• Heuristics and metaheuristics
• Cognitive science
• Conclusions
3 / 31
Outline
• Background
• Challenges in ARN
• Heuristics and metaheuristics
• Cognitive science
• Conclusions
4 / 31
Background
• Problem 𝑃 = 𝑆, 𝐼, 𝐴, 𝐺
• 𝑆: set of states
• 𝐼 ⊆ 𝑆: set of initial states
• 𝐴: set of possible actions 𝑎 ∈ 𝐴
• 𝐺 𝑠 is a Boolean goal test
• Problem domain = set of related problems that share some characteristic
• Path = a finite ordered sequence of interleaved states and actions 𝑠1, 𝑎1, 𝑠2, 𝑎2, … , 𝑠𝑟−1, 𝑎𝑟−1, 𝑠𝑟
• Solution = a path where 𝑠1 ∈ 𝐼 and 𝐺 𝑠𝑟 = 𝑇𝑟𝑢𝑒
• Optimal solution has minimum path cost
• Satisfactory solution is good enough
5 / 31
Path planning
• Plan = path proved to be a solution before execution
• Path planning = search for a plan that minimizes travel time, travel distance, or resource consumption
• Path quality = value of a path based on domain-specific criteria
Initial state
Goal state
Solution
Satisfactory solutions
Optimal solution?
6 / 31
Outline
• Background
• Challenges in ARN
• Heuristics and metaheuristics
• Cognitive science
• Conclusions
7 / 31
Challenges in ARN (1)
• Path planning = how do I get to where I need to go?
• Localization = where am I in the environment? How do I detect my pose 𝑥, 𝑦, 𝜃 despite sensor error?
• Mapping = what does the environment look like?
• Obstacle avoidance = how do I get around obstructions in the environment?
• Motion control = how do I account for error from my actuators?
NP-Hard [Canny, 1988]
Simultaneous localization and mapping (SLAM) [Durrant, 2006]
Reactive and online
Handled by hardware
8 / 31
• Current state-of-the-art for implemented ARN systems on commercial robots = SLAM with A* search
• Lack of standard testbed and performance metrics no conclusive evidence that any approach is best
• Since sub-optimal solutions must suffice many other approaches have been investigated
Challenges in ARN (2)
• Real world vs. simulation
• Environmental issues = observability, multiagency, dynamism, continuity, and modality
[Bartel, 2007] [Zhou, 2014]
9 / 31
Outline
• Background
• Challenges in ARN
• Heuristics and metaheuristics
• Cognitive science
• Conclusions
10 / 31
Heuristics and metaheuristics
• Heuristic = efficient strategy that can often solve problems, typically specific to a problem domain [Pearl, 1984]
• Metaheuristic = a broadly applicable and non-domain specific technique that uses a heuristic strategy to obtain satisfactory solutions [Glover, 2006]
• Hybrid metaheuristic = 2 metaheuristics or metaheuristic + another method [Blum, 2011]
• No guarantee that an optimal solution will be found in finite time satisfactory solutions suffice in path planning for ARN
• Candidate = path found during search
• Neighbor of a candidate = path with one change to the candidate’s action sequence
11 / 31
changed added removed
Local Global
Hill climbing (HC)
Steepest-ascent HC
Random search
HC with random restarts
Genetic algorithm
Genetic programming
Evolutionary programming
Differential evolution
Biogeography-based optimization
Particle swarm optimization
Bat algorithm
Cuckoo search algorithm
Gravitational search
Harmony search
Ant colony optimization
Artificial bee colony
Firefly algorithm
Memetic algorithm
Single solution
Population
Locality of search
Number of candidates considered
Simulated annealing
Tabu search
Heuristic-based methods
Taxonomy of search methods
Both
Metaheuristics
Evolutionary algorithms
Swarm algorithms
Other metaheuristics 12 / 31
[Talbi, 2009]
Local Global
Hill climbing (HC)
Steepest-ascent HC
Random search
HC with random restarts
Genetic algorithm
Genetic programming
Evolutionary programming
Differential evolution
Biogeography-based optimization
Particle swarm optimization
Bat algorithm
Cuckoo search algorithm
Gravitational search
Harmony search
Ant colony optimization
Artificial bee colony
Firefly algorithm
Memetic algorithm
Single solution
Population
Locality of search
Number of candidates considered
Simulated annealing
Tabu search
Heuristic-based methods
Taxonomy of search methods
Both
Metaheuristics
Evolutionary algorithms
Swarm algorithms
Other metaheuristics 13 / 31
[Talbi, 2009]
Heuristic-based methods
• Random search
• Pure exploration
• Hill climbing
• Pure exploitation of learned knowledge
• Hill climbing with random restarts
• Random exploration with periods of exploitation
• Hill climbing with steepest ascent
• Exploitation of learned knowledge with limited exploration around candidate
• Fast but highly susceptible to premature convergence to local optima
14 / 31
Local Global
Hill climbing (HC)
Steepest-ascent HC
Random search
HC with random restarts
Genetic algorithm
Genetic programming
Evolutionary programming
Differential evolution
Biogeography-based optimization
Particle swarm optimization
Bat algorithm
Cuckoo search algorithm
Gravitational search
Harmony search
Ant colony optimization
Artificial bee colony
Firefly algorithm
Memetic algorithm
Single solution
Population
Locality of search
Number of candidates considered
Simulated annealing
Tabu search
Heuristic-based methods
Taxonomy of search methods
Both
Metaheuristics
Evolutionary algorithms
Swarm algorithms
Other metaheuristics 15 / 31
[Talbi, 2009]
Single-solution metaheuristics
• Simulated annealing improves hill climbing [Kirkpatrick, 1983; Černỳ, 1985]
• Occasionally selects candidates with lower quality
• Changes the balance of exploration and exploitation by reducing the probability of selecting a lower quality candidates over time
• Tabu search also selects lower quality candidates [Glover, 1989; 1990]
• Prevents return to recently visited candidates
• Intentionally limits exploitation to increase exploration
• Advantages: efficient and can control memory requirements
• Disadvantages: highly dependent on initialization and parameter settings, may be slow
• Applications to path planning are rare and simulated in simplistic, unrealistic environments
16 / 31
Local Global
Hill climbing (HC)
Steepest-ascent HC
Random search
HC with random restarts
Genetic algorithm
Genetic programming
Evolutionary programming
Differential evolution
Biogeography-based optimization
Particle swarm optimization
Bat algorithm
Cuckoo search algorithm
Gravitational search
Harmony search
Ant colony optimization
Artificial bee colony
Firefly algorithm
Memetic algorithm
Single solution
Population
Locality of search
Number of candidates considered
Simulated annealing
Tabu search
Heuristic-based methods
Taxonomy of search methods
Both
Metaheuristics
Evolutionary algorithms
Swarm algorithms
Other metaheuristics 17 / 31
[Talbi, 2009]
Evolutionary algorithms (1)
• Inspired by Darwinian principles
• Genetic algorithm [Manikas, 2007]
• Evolves a population of candidates with reproduction, crossover, mutation, and selection
• Fitness function evaluates quality of candidates
• Genetic programming = candidates are computer programs [Koza, 1992]
• Evolutionary programming [Fogel, 1999]
• Candidates are value assignments to parameters of a single program
• The next generation is composed of the best candidates selected from among the parents and children
• Differential evolution [Storn, 1997]
• Candidates are vectors of real numbers that are function parameters
• Recombination incorporates a third population member during reproduction
18 / 31
Evolutionary algorithms (2)
• Applications to path planning• Single robot and unmanned aerial vehicle (UAV)• Multi-robot and multi-UAV• Multi-objective• Addition of domain knowledge
• Advantages: incorporates exploitation/hill-climbing through selection, and exploration/randomization through mutation
• Disadvantages:• Parameters must be hand tuned• Can be computationally and memory intensive
19 / 31
Local Global
Hill climbing (HC)
Steepest-ascent HC
Random search
HC with random restarts
Genetic algorithm
Genetic programming
Evolutionary programming
Differential evolution
Biogeography-based optimization
Particle swarm optimization
Bat algorithm
Cuckoo search algorithm
Gravitational search
Harmony search
Ant colony optimization
Artificial bee colony
Firefly algorithm
Memetic algorithm
Single solution
Population
Locality of search
Number of candidates considered
Simulated annealing
Tabu search
Heuristic-based methods
Taxonomy of search methods
Both
Metaheuristics
Evolutionary algorithms
Swarm algorithms
Other metaheuristics 20 / 31
[Talbi, 2009]
Swarm algorithms
• Simulates crowd behavior of organisms
• Ant colony optimization [Dorigo, 2006]
• Indirect communication when foraging
• Individual ants use local search
• Pheromones indicate candidate desirability
• Artificial bee colony [Karaboga, 2007]
• Three groups: employed bees, onlookers, and scouts
• Combines local search and global search
• Particle swarm optimization [Kennedy, 1995]
• Birds in search of food as particles that move through search space
• Direction of each particle’s local search is influenced by other particles
• Other approaches: firefly algorithm, bat algorithm, cuckoo search
21 / 31
Swarm algorithms comparison
22 / 31
Algorithm Individuals Type of search Communication mechanism
Ant colonyoptimization
ants hill climbing pheromone strength
Artificial bee colony
bees hill climbing and random search
dance intensity
Particle swarm optimization
particles hill climbing velocity of global best position
Firefly algorithm
fireflies hill climbing flashing brightness and distance
Bat algorithm bats hill climbing and random search
echolocation frequency, loudness, and rate of emissions
Cuckoo search cuckoos random search none
Path planning with swarm algorithms
• Applications• Single robot, UAV, and underwater• Multi-robot and multi-UAV• Multi-objective• Addition of domain knowledge• Dynamic obstacles
• Advantages: communication mechanisms allow local information to influence entire population
• Disadvantages:• Parameters must be hand tuned• Can be computationally and memory intensive
23 / 31
Hybrid metaheuristics
• Categorized by
• Type of hybrid: dual (2 metaheuristics) or metaheuristic + another method
• Approach to ARN: blend components into a new system or use them separately for different tasks
• Applications to path planning
• Mostly in static, simulated environments
• Dual hybrids typically evolutionary algorithm + swarm algorithm
• Other methods include probabilistic roadmap, chaotic search, fuzzy logic, and artificial potential field
• Advantages: tries to resolve shortcomings of individual methods by combining multiple methods
• Disadvantages:
• Difficult to compare approaches
• Parameters must be hand tuned
• Increased computational complexity
24 / 31
Outline
• Background
• Challenges in ARN
• Heuristics and metaheuristics
• Cognitive science
• Conclusions
25 / 31
Cognitive science
• Multidisciplinary field that studies the mind and intelligence [Friedenberg, 2011]
• Spatial cognition = subfield of cognitive science that studies navigation and wayfinding
• Decision making = agent selects among a set of alternatives
• Reasoning = agent draws a conclusion from information to solve a problem or make a decision
• People exhibit goal-directed behavior
• People use heuristics to make decisions, especially when faced with limited time, knowledge, or computational power [Gigerenzer, 1999]
• People use case-based reasoning
• People employ hierarchical organization of spatial memory [Wiener, 2003]
26 / 31
Human navigation
27 / 31
Strategies for human navigation
What? Spatial knowledge: visual cues (landmarks), frame of reference (egocentric vs allocentric), spatial orientation, path integration, route knowledge, survey knowledge
Internal representation: cognitive map [Golledge, 1999]
Other knowledge: external information, previous knowledge and experience
Who? Demographics: age and gender Spatial abilities: sense of direction
Where? Simulated: virtual environments Real world: indoor and outdoor environments
When? Before navigation: path planning, search and selection
During navigation: following a plan, or decision making without a plan
Why? Adapted to overall goal or current task [Holscher, 2011]
Adapted to overall environment or current state
How? Sense: proprioception, vision and hearing
Decision and action: brain activity and muscle activation
How much? Cognitive economy
Cognitive models
• Cognitive models simulate observed human behavior with a computational system or algorithm
• Application of cognitive models to ARN• Build a hierarchical representation of the environment similar to a
cognitive map [Kuipers, 2000]
• Represent spatial knowledge with a graph• Shift between navigation strategies using a heuristic• Bayesian approach + artificial neural network to learn topological
maps and landmarks [Thrun, 1998]
• Learn abstract representations of the environment and use multiple heuristics to make decisions [Epstein, 2015]
• Advantages: exploit human knowledge and strategies
• Disadvantages:• No one model has simulated all observed human wayfinding behavior
or employed all their strategies• Difficult to account for all individual differences among people
28 / 31
Outline
• Background
• Challenges in ARN
• Heuristics and metaheuristics
• Cognitive science
• Conclusions
29 / 31
Conclusions
• Most approaches have been evaluated in simplified simulated environments that underestimate the hardware challenges of physical robots and reduce computational complexity
• But people successfully navigate in the real world already so there is room for improvement inspired by human behavior
• Ultimately, autonomous robot navigation systems must
• operate in the real world
• contend with observability, multiagency, dynamism, continuity, and modality issues
• Future work should address these challenges in a real-world environment
• Potential future work could create more robust ARN systems by a synergy between metaheuristics and cognitive models
30 / 31
Thank You
31 / 31