Bayesian Maps: probabilistic and hierarchical models for ... · tions, they usually have no trouble navigating from the sofa to the bed, or learning to do so when moving in a new

Bayesian Maps: probabilistic and hierarchical

models for mobile robot navigation

Julien Diard1 and Pierre Bessière2

1 Laboratoire de Psychologie et NeuroCognition CNRS UMR 5105,Université Pierre Mendès France, BSHM, BP 47,38040 Grenoble Cedex 9, France,[email protected]

2 Laboratoire GRAVIR / IMAG - CNRS INRIA Rhône-Alpes,655 avenue de l’Europe, 38330 Montbonnot Saint Martin, France,[email protected]

1 Introduction

Imagine yourself lying in your bed at night. Now try and answer these ques-tions: Is your body parallel or not to the sofa you have, two rooms away fromyour bedroom? What is the distance between your bed and the sofa? If weexcept cases like rotating beds, people who actually sleep in their sofas, ortiny apartments, these questions are usually non-trivial, and answering themrequires abstract thought. And if pressed to answer quickly, so as to forbid theuse of abstract geometry learned in high school, the reader will very probablygive wrong answers.

However, if people had the same representations of their environment thatroboticians usually provide to their robot, answering these questions wouldbe very easy. The answers would come quickly, and they would certainly becorrect. Indeed, robotic representations of space are usually based on large-scale, accurate, metric cartesian maps. This allows judgment of parallelismand estimations of distances to be straightforward.

On the other hand, even though humans have difficulties with these ques-tions, they usually have no trouble navigating from the sofa to the bed, orlearning to do so when moving in a new apartment. Robots have more diffi-culties in the same situation. In most robotic mapping approaches, the acqui-sition of a precise, and, more importantly, accurate map of the environmentis a prerequisite to solving navigation tasks. This is still a difficult and openissue in the general case.

Therefore, there appears to be a discrepancy in representations of spacebetween the ones we provide usually to the robots we build and program, andthe representations of space humans, or animals, use. Indeed, the nature, num-ber, and possible interplay of the spatial representations involved in human or

234 Julien Diard and Pierre Bessière

animal navigation processes are still an open question in life sciences. Thereis also a discrepancy in the difficulty of navigation tasks currently solved bystate-of-the-art robots and the navigation task solved very easily by humansor animals.

We believe studying the difference between robotics and life sciences mod-els of navigation can be very fruitful, both for modeling better robots andbetter understanding animal navigation. That is the topic of this paper.

We first propose a quick overview of navigation models, both in roboticsand in biology. We will first focus, more precisely, on probabilistic approachesto navigation and mapping in robotics. These approaches include – but arefar from limited to – Kalman Filters Leonard et al. [1992], Markov Localiza-tion models Thrun [2000], (Partially or Fully) Observable Markov DecisionProcesses Kaelbling et al. [1998], and Hidden Markov Models Rabiner andJuang [1993]. We will here assume that the reader has some familiarity withthese approaches. We will show how these methods differ from most modelsof human or animal navigation, which we give a brief introduction to: whereasrobotics approaches mostly rely on large-scale monolithic representations ofspace, models of animal navigation, right from the start, assume hierarchiesof representations. We thus then describe hierarchical approaches to roboticmapping.

Indeed, in this domain of probabilistic modeling for robotics, hierarchicalsolutions are currently flourishing. However, we will argue that the main phi-losophy used by all these approaches is to try to extract, from a very complexbut intractable model, a hierarchy of smaller models. Of course, automaticallyselecting the relevant decomposition of a problem into sub-problems is quite achallenge – this challenge being far from restricted to the domain of navigationfor robots facing uncertainties.

We propose, in this paper, to pursue an alternate route. We investigatehow, starting from a set of simple probabilistic models, one can combine themfor building more complex models. The goal of this paper is therefore topresent a new formalism for building models of the space in which a robot hasto navigate (the Bayesian Map model), and a method for combining such mapstogether in a hierarchical manner (the Abstraction operator). This formalismallows for a new representation of space, in which the final program is builtupon many imbricate models, each of them deeply rooted into lower levelsensorimotor relationships.

For brevity, this paper will discuss neither of the learning methods that canbe included into Bayesian Maps Simonin et al. [2005], nor of other operatorsfor merging Bayesian Maps (the Superposition operator Diard et al. [2005], theSensorimotor Integration Operator Gilet et al. [2006]). More details missingfrom the current paper can also be found in Diard’s Ph.D. thesis Diard [2003].

The rest of this paper is organized as follows. Section 2 gives a quickoverview of the most prominent models of navigation and representation oflarge-scale space, first from a robotics point of view, then from a life sciencespoint of view. Section 3 proposes a comparison of the main characteristics

Bayesian Maps 235

of the models, an analysis of their strength and weaknesses, which argues infavor of the need for hierarchical and modular probabilistic models of naviga-tion. We then introduce our contribution to the domain, the Bayesian Mapformalism (Section 4), and one of the operators we defined for combiningBayesian Maps, the Abstraction Operator (Section 5). Finally, we report inSection 6 a series of robotic experiments where we apply the Bayesian Mapmodel and the Abstraction Operator on a Koala mobile robot, in a proof ofconcept experiment.

2 Navigation models in robotics and biology

We focus this brief review of existing models of navigation skills, both inrobotics and life sciences. Because the literature is so large in robotics con-cerning the representation of space, we focus here on probabilistic approachesto mapping. In life sciences, we describe some of the more prominent theoret-ical models of large-scale navigation in humans and animals, focusing on theirhierarchical nature.

2.1 Probabilistic models of navigation and mapping

There is currently a wide variety of models in the domain of probabilisticmobile robot programming. These approaches include Kalman Filters (KF,Leonard et al. [1992]), Markov Localization models (ML, Thrun [2000]),(Partially or Fully) Observable Markov Decision Processes (POMDP, MDP,Boutilier et al. [1999]), Hidden Markov Models (HMM, Rabiner and Juang[1993]), Bayesian Filters (BFs), or even Particle Filters (PFs). The literaturecovering these models is huge: for references that present several of them atonce, giving unifying pictures, see Murphy [2002], Roweis and Ghahramani[1999], Smyth et al. [1997]. Some of these papers define taxonomies of theseapproaches, by proposing some order which helps classifying them into fam-ilies. One such taxonomy is presented Fig. 1 (from Diard et al. [2003]). It isbased on a general-to-specific ordering: for example, it shows that DynamicBayesian Networks (DBN) are a specialization of the Bayesian Network for-malism, tailored for taking into account time series. We can see Fig. 1 subtreesthat correspond to different specialization strategies. In the remainder of thissection, we will focus on the Markov Localization subtree, which correspondsto specializing DBNs by the choice of a four variable model.

The ML model is basically a HMM with an additional action variable.Indeed, it seems relevant in the robotic programming domain, since robotsobviously can affect their states via motor commands. The stationary modelof a HMM is basically the decomposition

P (P t Lt Lt−1) = P (Lt−1)P (Lt | Lt−1)P (P t | Lt), (1)


Fig. 1. Some common probabilistic modeling formalisms and their general-to-specific partial ordering (reprinted from Diard et al. [2003]).

where P t is a perception variable, Lt and Lt−1 are state variables or, more pre-cisely in our navigation case, location variables at time t and t−1. P (Lt | Lt−1)is commonly called the transition model and P (P t | Lt) is referred to as theobservation model. Starting from this structure, the action variable At is usedto refine the transition model P (Lt | Lt−1) into P (Lt | At Lt−1), which iscalled the action model. Thus the ML model is sometimes referred to as theinput-output HMM model. Due to the generality of the BRP formalism, themodel for Markov Localization can be cast into a BRP program. This is shownFig. 2.

Pro

gra

m

8

>>>>>>>>>>>>>>>>>>>>>><

>>>>>>>>>>>>>>>>>>>>>>:

Des

crip

tion

8

>>>>>>>>>>>>>>>>>><

>>>>>>>>>>>>>>>>>>:

Spec

ifica

tion

8

>>>>>>>>>>>>>><

>>>>>>>>>>>>>>:

Relevant Variables:P t : perception variableLt : discrete location variable at time tLt−1 : discrete location variable at time t − 1At : action variableDecomposition:P (At Lt Lt−1 Lt) =

P (Lt | At Lt−1)P (P t | Lt)P (At)P (Lt−1)Parametric Forms:usually, matrices or particles

Identification:any

Question:localization P (Lt | At P t)

Fig. 2. The Markov Localization stationary model expressed as a BRP.

The ML model is mostly used in the literature to answer the questionP (Lt | At P t), which estimates the state of the robot, given the latest motor

Bayesian Maps 237

commands and sensor readings. When this state represents the position of therobot in its environment, this amounts to localization. When this stationaryML model is replicated over time in order to estimate the positions L0:T ofthe robot over a long series of time steps, it can be shown that an iterativelocalization procedure exists, that localizes the robot by just updating thelast position estimate in view of the latest motor commands At and sensorreadings P t. This justifies, in this presentation, the focus on the stationarymodel.

The ML model has been successfully applied in a range of robotic ap-plications, the most notable examples being the Rhino (Thrun et al. [1998],Burgard et al. [1999]) and Minerva (Thrun et al. [1999a,b]) robotic guides.The most common application of the ML model is the estimation of the po-sition of a robot in an indoor environment, using a fine-grained metric gridas a representation. In other words, in the model of Fig. 2, the state variableis very frequently the pose of the robot, i.e., a pair of x, y discrete Cartesiancoordinates for the position, and an angle θ for the orientation of the robot.Assuming a grid cell size of 50 cm, an environment of 50 × 50 m2, and a 5̊angle resolution entails a state space of 720,000 states.

Using some specialized techniques and assumptions, it is possible to makethis model tractable.

For example, the forms of the probabilistic model can be implemented us-ing sets of particles. These approximate the probability distributions involvedin Fig. 2, which leads to an efficient position estimation. This specialization iscalled the Monte Carlo Markov Localization model (MCML, Fox et al. [1999]).

Another possibility is to use a Kalman Filter as a specialization of the MLmodel, in which variables are continuous. The action model P (Lt | At Lt−1)and the observation model P (P t | Lt) are both specified using Gaussian lawswith means that are linear functions of the conditioning variables. Due tothese hypotheses, it is possible to analytically solve the inference problem toanswer the localization question. This leads to an extremely efficient algorithmthat explains the popularity of Kalman Filters.

2.2 Biologically-inspired models

All the approaches mentioned in the preceding Section are based on the clas-sical view of robotic navigation, which is inherited from marine navigation.In this view, solving a navigation task basically amounts to answering se-quentially the questions of Levitt and Lawton: “Where am I?”, “Where arethe other places with respect to me?”, and “How do I get to other placesfrom here?” Levitt and Lawton [1990], or alternatively, those of Leonard andDurrant-Whyte: “Where am I?”, “Where am I going?”, and “How should I getthere?” Leonard and Durrant-Whyte [1991].

While a valid first decomposition of the navigation task into subtasks,these questions have usually led to models that require a global model of theenvironment, which allows the robot to localize itself (the first questions), to


infer spatial relationships between the current recognized location and otherlocations (the second questions), and to plan a sequence of actions to movein the environment (the third questions). These skills amount to the first twophases of the “perceive, plan, act” classical model of robotic control.

Very early in their analysis, biomimetic models of navigation dispute thisclassical view of robotic navigation. Indeed, when studying living beings, theexistence of such a unique and global representation that would be used tosolve these three questions is very problematic. This seems obvious even forsimple animals like bees and ants. For instance, the desert ant cataglyphisoutdoor navigation capabilities are widely studied, and rely on the use of thepolarization patterns of the sky Lambrinos et al. [2000]. But it is clear thatsuch a strategy is useless for navigating in their nest; this calls for anothernavigation strategy, and another internal model. The existence of a uniquerepresentation it is also doubtful for human beings. The navigation capabil-ities of humans are based on internal models of their environment (cognitivemaps), but their nature, number and complexities are still largely debated (seefor instance Berthoz [2000], Redish and Touretzky [1997], Wang and Spelke[2002], for entry points into the huge literature associated to this domain).

As a consequence, biomimetic approaches assume, right from the start,the existence of multiple representations, most often articulated in a hierar-chical manner. We now give a brief review of some theories from that domain,focusing on their hierarchical components.

Works by Redish and Touretzky

Works by Redish and Touretzky address the issue of the role of the hippocam-pus and para-hippocampal populations in rodent navigation, focusing on thewell-studied place cells and head direction cells. They proposed a conceptualmodel Touretzky and Redish [1996] and discussed its anatomical plausibil-ity Redish and Touretzky [1997]. Their hierarchical conceptual model consistsof four spatial representations (place code, local view, path integrator andhead direction code), supplemented by two components called the referenceframe selection subsystem and the goal subsystem.

Place codes are local representations tied to one or several landmarks orgeometric features of the environment. When the environment of the animalgets large or structured, several place codes may be used to describe thisenvironment, each place code representing a part of the environment. Forinstance, Gothard et al. Gothard et al. [1996] found different place codes for arat navigating in an environment containing a goal and a starting box. Theyidentified three independent place codes: one tied to the room, one to thegoal, and one to the box. These effectively provide representations of sectionsof the environment: cells tuned to the box frame were only active when therat was in or around the box, cells tuned to the goal only responded when therat was near the goal, cells tuned to the room were active at other times (i.e.when the rat was not near the box or the goal).

Bayesian Maps 239

Fig. 3. The Parallel Map Theory components (reprinted from Jacobs [2003], per-mission requested)

The reference frame selection component is responsible for selecting theappropriate place code for navigating in the environment. In the above exam-ple, this means that it is responsible for selecting, at any given time, whichplace code should be active.

This theory thus proposes an account of the low-level encoding of spacein central nervous systems of animals using a two-layer hierarchy of models.The low-level layer consists of a series of place codes describing portions of theanimal environment, under the hierarchical supervision of a larger-space-scalemodel.

Computational models of the low-level component of this hierarchy, (i.e.place cells and head-direction cells) abound in the literature (e.g. Hartley andBurgess [2002]), whereas the reference frame selection component, to the bestof our knowledge, has yet to be mathematically defined.

Works by Jacobs and Schenk

Jacobs and Schenk proposed a new theory of how the hippocampus encodesspace Jacobs and Schenk [2003], Jacobs [2003]. This theory is called the Paral-lel Map Theory (PMT), and defines a hierarchy of navigation representationsmade of three components and two layers: see Fig. 3.

The bearing map is the first, low-level, component. It is a single mapbased on several directional cues such as intersecting gradients. It provides alarge-scale two-dimensional reference frame, allowing for large-scale navigationskills, simply using gradient ascent or descent.

The sketch maps are the second components of the low-level layer of thehierarchy. They encode small-scale fine-grained representations of the rela-tionship of landmarks close to each other (positional cues). This creates local


representations, which can be used for controlling precisely the position, andthus, solving precise, small-scale navigation tasks.

Finally, the integrated map is the third, high-level, component: it is con-structed from the bearing map and several sketch maps. It consists of a unifiedmap of large-scale environments, where the local sketch maps are cast into thelarge-scale reference frame of the bearing map. This provides the means toinfer large-scale spatial relationship between the local, metric representationsof the sketch maps, thus allowing computation of large-scale shortcuts anddetours.

To the best of our knowledge, the papers by Jacobs and Schenk do notprovide computational models of these different components. Instead, theymainly focus on the anatomical and phylogenetic plausibility of their concep-tual model. This provides a lot of experimental predictions concerning possibleimpairments resulting from lesions.

Works by Wang and Spelke

These authors dispute the idea that enduring, allocentric, and large-scale rep-resentations of an environment should be the main theoretical tool used forinvestigating navigation in humans and animals. Indeed, the Cognitive Mapconcept, introduced by Tolman in 1948, is still controversial Tolman [1948].Instead, Wang and Spelke argue that many navigation capabilities in animalscan be explained by dynamic, egocentric representations that cover a limitedportion of the environment Wang and Spelke [2000, 2002]. Such representa-tions can be studied in animals which are far simpler than humans, like desertants for instance Lambrinos et al. [2000].

Studies on these animals have identified three subsystems: a path integra-tion system, a landmark based navigation system, and a reorientation system.This last component is not hierarchically related to the other two, as it ismainly responsible for the resetting of the path integration system when theanimal gets disoriented. However, the first two component show a strong hier-archical relation. Indeed, it has been shown that the landmark based strategyis hierarchically higher in the cognitive mechanisms of insects and rodents.It also appears that, in the sudden absence of landmarks after learning apath, animals rely on the path integration encoding as a "back up" encodingStackman et al. [2002], Stackman and Herbert [2002].

This model is somewhat different from the previous studies, as it focuseson defining a hierarchy of skills of navigation, instead of hierarchies of repre-sentations of space, as in the PMT or studies of the hippocampal and para-hippocampal areas.

Works by Kuipers, Franz, Trullier

The hierarchies of models proposed in the biomimetic robotic literature(Kuipers [1996], Trullier et al. [1997], Franz and Mallot [2000], Kuipers [2000],

Bayesian Maps 241

Victorino and Rives [2004]) have several aspects: they are hierarchies of in-creasing navigation skills, but also of increasing scale of the represented en-vironment, of increasing time scale of the associated movements, and of in-creasing complexity of representations. This last aspect means that topologicrepresentations, which are simple, come at a lower level than global metricrepresentations, which are arguably more complex to build and manipulate.This ordering stems from the general observation that animals that are ableto use shortcuts and detours between two arbitrary encoded places (skills thatrequire global metric models) are rather complex animals, like mammalians.These skills seem to be mostly absent from simpler animals, like invertebrates.

The resulting proposed hierarchies show a striking resemblance. We presentthe salient and common features of these hierarchies by summarizing the Spa-tial Semantic Hierarchy (SSH) proposed by Kuipers Kuipers [1996, 2000]. It is,to the best of our knowledge, the only biomimetic approach that was appliedto obtain a complete and integrated robotic control model.

The SSH essentially consists of four hierarchical levels: the control level,the causal level, the topological level, and the metrical level.

The control level is a set of reactive behaviors, which are control lawsdeduced from differential equations. These behaviors describe how to movethe robot for it to reach an extremum of some gradient measure. This ex-tremum can be zero-dimensional (a point in the environment), in which caseit is called a locally distinctive state. The associated behavior is called a hill-climbing law. The extremum can also be one-dimensional (a line or curve inthe environment), in which case the behavior is called a trajectory-followinglaw. Provided that any trajectory-following law guarantees arriving in a placewhere a hill-climbing law can be applied, then alternating laws of both typesdisplace the robot in a repetitive fashion. This solves the problem of the ac-cumulation of odometry errors. The control level is also referred to as theguidance level Trullier et al. [1997], Franz and Mallot [2000].

Given the control level, the environment can be structured and summa-rized by the locations of locally distinctive states and the trajectories used togo from one such state to another. This abstraction takes place in the causallevel, which is the second level of the hierarchy of representations. Unlike thecontrol level, it allows the robot to memorize relationships between placesthat are outside of the current perceptive horizon (what is part of the way-finding capabilities, in other terminologies Trullier et al. [1997], Franz andMallot [2000]). To do so, Kuipers abstracts locally distinctive places as viewsV , the application of lower level behaviors as actions A, and defines schemasas tuples 〈V, A, V ′〉 (expressed as first-order logic predicates). The schemashave two meanings. The first is a procedural meaning: “when the robot isin V , it must apply action A.” This aspect of the schemas is equivalent tothe recognition-triggered response level of the other hierarchies Trullier et al.[1997], Franz and Mallot [2000], or to the potential field approaches, or toother goal-oriented methods. But the second meaning of schemas is a declar-ative one, where 〈V, A, V ′〉 stands for: “applying action A from view V brings


eventually the robot at view V ′.” This allows using the schemas for predictionof future events, or in a planning process, for example.

The goal of the topological level is to create a globally consistent represen-tation of the environment, as structured by places, paths and regions. Theseare extracted from lower level schemas, by an abduction process, which createsthe minimum number of places, paths and regions so as to be consistent withthe known schemas. Places are zero-dimensional parts of the environment,which can be abstractions of lower level views, or abstractions of regions (forhigher-level topological models). Paths are one-dimensional, are oriented, andcan be built upon one or more schemas. Finally, regions are two-dimensionalsubspaces, delimited by paths. It must be noted that, since the accumulationof odometry error problem was dealt with at the control level, building a glob-ally consistent topological representation (i.e. solving the global connectivityproblem) is much easier. To do so, Kuipers proposes an exploration strategy,the rehearsal procedure, which, unfortunately, relies on a bound on the ex-ploration time, and is not well suited for dynamic environments. The placesand paths of the obtained topological representation can be used for solvingplanning queries using classical graph-searching algorithms.

The last level is the metrical level, in which the topological graph is castinto a unique global reference frame. For reasons outlined above (and detailedin Kuipers [2000]), this level is considered as a possibility, not a prerequisitefor solving complex navigation tasks. If the sensors are not good enough tomaintain a good estimation of the Cartesian coordinates of the position, forinstance, it is still possible to use the topological model for acting in the envi-ronment – while shortcuts and detours are not possible. Indeed, few roboticssystems implementing biomimetic models include the metrical level Franz andMallot [2000], Trullier et al. [1997].

3 Theoretical comparison

3.1 What mathematical formalism?

A major drawback of the biomimetic models presented previously is that theyseldom are defined as computational models. In other words, they give generalframeworks for understanding animal navigation, but do not make simulationsof these models operational. The notable exception being the SSH model,which not only defines layers in a hierarchy of space representations, but alsodefines each of them mathematically.

However, the SSH model uses a variety of formalisms for expressing knowl-edge at different layers of the hierarchy: differential equations and their so-lutions for the control level, first-order logic and deterministic algorithms forhigher-level layers of the hierarchy. This makes it difficult to theoreticallyjustify the consistency and correctness of the mechanisms for communication

Bayesian Maps 243

between the layers of the hierarchy. In some cases, it even limits and con-straints the contents of the layers: for instance, the SSH model requires thatthe behaviors of the control level guarantee that the robot reaches the neigh-borhood of a given locally distinctive state. In our view, this constraint isbarely acceptable for any kind of realistic robotic scenario. Consider dynamicenvironments: how to guarantee that a robot will reach a room, if the dooron the way can be closed?

We take as a starting point of our analysis that the best formalism forexpressing incomplete knowledge and dealing with uncertain information isthe probabilistic formalism Bessière et al. [1998], Jaynes [2003]. This gives usa clear and rigorous mathematical foundation for our models. The probabil-ity distributions are our unique tool for the expression and manipulation ofknowledge, and in particular, for the communication between submodels. Wewill thus argue in favor of hierarchical probabilistic models.

3.2 Hierarchical probabilistic models

This idea is not a breakthrough: in the domain of probabilistic modelingfor robotics, hierarchical solutions are currently flourishing. The more activedomain in this regard is decision theoretic planning: one can find variants ofMDPs that accommodate hierarchies or that select automatically the partitionof the state-space (see for instance Hauskrecht et al. [1998], Lane and Kaelbling[2001], or browse through the references in Pineau and Thrun [2002]). Moreexceptionally, one can find hierarchical POMDPs Pineau and Thrun [2002].The current work can also be related to Thrun’s object mapping paradigmThrun [2002], in particular concerning the aim of transferring some of theknowledge the programmer has about the task, to the robot.

Some hierarchical approaches outside of the MDP community include Hier-archical HMMs and their variants (see Murphy [2002] and references therein),which, unfortunately, rely on the notion of final state of the automata. An-other class of approaches relies on the extraction of a graph from a proba-bilistic model, like for example a Markov Localization model Thrun [1998],or a MDP Lane and Kaelbling [2002]. Using such deterministic notions isinconvenient in a purely probabilistic approach, as we are pursuing here.

Moreover, the main philosophy used by all the previous approaches is totry to extract, from a very complex but intractable model, a hierarchy ofsmaller models (structural decomposition, see Pineau and Thrun [2002]).

Again, this comes from the classical robotic approach, where the processof perception (in particular, the localization) is assumed to be independent ofthe processes of planning and action. A model such as the ML model (Fig. 2)is only concerned with localization, not control: therefore its action variableAt is actually only used as an input to the model. In this view, a pivotalrepresentation is used between the perception and planning subproblems. Itis classically assumed that the more precise this pivotal model, the better.Unfortunately, when creating integrated robotic applications, dealing with


both the building of maps and their use is necessary. Some authors haverealized, at this stage, that their global metric maps were too complex tobe easily manipulated. Therefore, they have tried to degrade their maps, thatwas so difficult to obtain initially, by extracting graphs from their probabilisticmodels, for instance Thrun [1998]. This problem is also the core of the roboticplanning domain, where the given description of the environment is assumed tobe an infinitely precise geometrical model. The difficulty is to discretize thisintractable, continuous model, into a finite model Latombe [1991], Kavrakiet al. [1996], Mazer et al. [1998], Svestka and Overmars [1998] (typically, inthe form of a graph).

3.3 Modular probabilistic models: toward Bayesian Maps

We propose to pursue an alternate route, investigating how, starting froma set of simple models, one can combine them for building more complexmodels. Such an incremental development approach allows us to depart fromthe classical “perceive, plan, act” loop, considering instead hierarchies builtupon many imbricate models, each of them deeply rooted into lower levelsensory and motor relationships.

The Bayesian Robotic Programming methodology offers exactly the formaltool that is needed to transfer information from one program to another in atheoretically rigorous fashion. Indeed, in Bayesian Robotic Programs, termsappearing in a description c1 can be defined as a probabilistic question toanother description c2. This creates a link between two descriptions, one beingused as a resource by another. Depending on the way questions are usedto link subprograms, several different operators can be created, each withspecific semantics: for instance, in the framework of behavior based robotics,Lebeltel has defined behavior combination, hierarchical behavior composition,behavior sequencing, sensor model fusion operators. He has also applied thesesuccessfully to realize a complex watchman robot behavior using a controlarchitecture involving four hierarchical levels Lebeltel et al. [2004].

This allows to solve a global robotic task problem by first, decomposingit into subproblems, then, writing a Bayesian Robot Program for each sub-problem, and finally, combining these subprograms. This method makes robotprogramming similar to structured computer programming. So far in our work,we let the programmer do this analysis: relevant intermediary representationscan be imagined, or copied from living beings. We propose to apply this strat-egy to the map-based navigation of mobile robots. The submodels can besubmaps, in the spatial sense (i.e. covering a part of the global environment),or in the subtask sense (i.e. modeling knowledge necessary for solving partof the global navigation task), or even in less familiar senses (e.g. modelingpartial knowledge from part of the sensorimotor apparatus).

Our approach is therefore based on a formalism for building models ofthe space in which a robot has to navigate, called the Bayesian Map model,

Bayesian Maps 245

that allows to build submodels which provide behaviors as resources. We alsodefine Operators for combining such maps together in a hierarchical manner.

4 The Bayesian Map formalism: definition

4.1 Probabilistic definition

A Bayesian Map c is a description that defines a joint distribution

P (P Lt Lt′ A | c),

where:

• P is a perception variable (the robot reads its values from physical sensorsor lower level variables),

• Lt is a location variable at time t,• Lt′ is a variable having the same domain than Lt, but at time t′ (without

loss of generality, let us assume t′ > t),• and A is an action variable (the robot writes commands on this variable).

For simplicity, we will assume here that all these variables have finite domains.The choice of decomposition is not constrained: any probabilistic depen-

dency structure can therefore be chosen here: see Attias [2003] for an exampleof how this leverage can lead to interesting new models. Finally, the definitionof forms and the learning mechanism (if any) are not constrained, either.

For a Bayesian Map to be useable in practice, we need the description to berich enough to generate behaviors. We call elementary behavior any questionof the form P (Ai | X), where Ai is a subset of A, and X a subset of theother variables of the map (i.e., not in Ai). A typical example consists ofthe probabilistic question P (A | [P = p] [Lt′ = l]): compute the probabilitydistribution over actions, given the current sensor readings p and the goal lto reach in the internal space of possible locations.

A behavior can be not elementary, for example if it is a sequence of el-ementary behaviors, or, in more general terms, if it is based on elementarybehaviors and some other knowledge (which need not be expressed in termsof maps).

For a Bayesian Map to be interesting, we will also require that it generatesseveral behaviors – otherwise, defining just a single behavior instead of a mapis enough. Such a map is therefore a resource, based on a location variablerelevant enough to solve a class of tasks: this internal model of the world canbe reified.

A “guide” one can use to “make sure” that a given map will generate usefulbehaviors, is to check if the map answers in a relevant manner the threequestions P (Lt | P ) (localization), P (Lt′ | A Lt) (prediction) and P (A | Lt Lt′)(control).


By “relevant manner”, we mean that these distributions have to be infor-mative, in the sense that their entropy is “far enough” of its maximum (i.e. the distribution is different from a uniform distribution). This constraintis not formally well defined, but it seems intuitive to focus on these threequestions. Indeed, the skills of localization, prediction and control are wellidentified in the literature as means to generate behaviors. Checking that theanswers to these questions are informative is a first step to evaluate the qualityof a Bayesian Map with respect to solving a given task.

Fig. 4 is a summary of the definition of the Bayesian Map formalism.P

rogra

m

8

>>>>>>>>>>>>>>>>>>>>>>><

>>>>>>>>>>>>>>>>>>>>>>>:

Des

crip

tion

8

>>>>>>>>>>>>>>>><

>>>>>>>>>>>>>>>>:

Spec

ifica

tion

8

>>>>>>>>>>>><

>>>>>>>>>>>>:

Relevant Variables:P : perception variableLt : location variable at time t

Lt′ : location variable at time t′, t′ > tA : action variableDecomposition:

AnyParametric Forms:Any

Identification:Any

Question:elementary behaviors: P (Ai | X), with Ai ⊆ A,

X ⊆“

{P, Lt, Lt′ , A} \ Ai

”

Fig. 4. The Bayesian Map model definition expressed in the BRP formalism.

4.2 Generality of the Bayesian Map formalism

We now invite the reader to verify that the Markov Localization model isindeed a special case of the Bayesian Map model by comparing Fig. 2 andFig. 4. Recall that Kalman Filters and Particle Filters are special cases ofMarkov Localization, as they add hypotheses over the choice of dependencystructure made by the Markov Localization model. This implies that KalmanFilters and Particle Filters also are special cases of Bayesian Maps.

Bayesian Maps can therefore accommodate many different forms, depend-ing on the needs or information at hand: for example, one Bayesian Mapcan be structured like a real valued Kalman Filter for tracking the angle anddistance to some feature when it is available. If that feature is not present,or in cases where the linearity hypotheses fail, we can use another BayesianMap, which need not be a Kalman Filter (for example, based on a symbolicvariable).

Bayesian Maps 247

Hierarchies built of Bayesian Maps (via the abstraction operator) can thusbe hierarchies of Markov Localization models, hierarchies of Kalman Filters,etc. Moreover, heterogeneous hierarchies of these models can be imagined: MLover KFs, or even n KFs and one ML model, which, in our view, would bea potential alternative to the solution of Tomatis et al. Tomatis et al. [2001,2003].

5 Putting Bayesian Maps together: definition of theAbstraction Operator and example

Having defined the Bayesian Map concept, we now turn to defining operatorsfor putting Bayesian Maps together. The one we present here is called theAbstraction of maps, it is defined Fig. 5, and commented in the rest of thissection.

Pro

gra

m

8

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>><

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>:

Des

crip

tion

8

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>><

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>:

Spec

ifica

tion

8

>>>>>>>>>>>>>>>>>>>>>>>>>>><

>>>>>>>>>>>>>>>>>>>>>>>>>>>:

Relevant Variables:

P =Vn

i=1

“

Pi ∧ Lti ∧ Lt′

i ∧ Ai

”

Lt : DLt = {c1, c2, . . . , cn}, kLt = n

Lt′ : DLt′ = {c1, c2, . . . , cn}, kLt′ = n

A : DA = {a1, a2, . . . , ak}, kA = kDecomposition:

P (P Lt Lt′ A) =

P (Lt)P (Lt′)P (A | Lt Lt′)Qn

i=1 P (Pi Lti Lt′

i Ai | Lt)Parametric Forms:P (Lt) = Uniform

P (Pi Lti Lt′

i Ai | [Lt = c])

=

if c = ci then P (Pi Lt

i Lt′

i Ai | ci)else Uniform

P (Lt′) = Uniform

P (A | Lt Lt′) = TableIdentification:

a priori programming or learning of P (A | Lt Lt′)Question:

P (Lt | P ) = 1Z1

Qn

i=1 P (Pi Lti Lt′

i Ai | Lt)

P (Lt′ | A Lt) = 1Z2

P (A | Lt Lt′)

P (A | Lt Lt′) = P (A | Lt Lt′).

Fig. 5. The abstraction operator definition expressed as a Bayesian Map.

As stressed above, in a Bayesian Map, the semantics of the location vari-able can be very diverse. The main idea behind the abstraction operator isto build a Bayesian Map c whose different locations are other Bayesian Maps


c1, c2, . . . , cn. The location variable of the abstract map will therefore take npossible symbolic values, one for each underlying map ci. Each of these mapswill be “nested” in the higher level abstract map, which justifies the use ofthe term “hierarchy” in our work. Recall that Bayesian Maps are designed forgenerating behaviors. Let us note a1, a2, . . . , ak the k behaviors defined in then underlying maps, with k ≥ n. In the abstract map, these behaviors canbe used for linking the locations ci. The action variable of the abstract mapwill therefore take k possible symbolic values, one for each behavior of theunderlying maps. In order to build an abstract map having n locations, theprogrammer will have to have previously defined n lower level maps, whichgenerate k behaviors. The numbers n and k are therefore small, and so theabstract map deals with a small internal space, having retained of each under-lying map only a symbol, and having “forgotten” all their details. This justifiesthe use of the name “abstraction” for this operator. But this “summary mech-anism” has yet to be described: that is what the perception variable P ofthe abstract map will be used for, as it will be the list of all the variablesappearing in the underlying maps: P = P1, L

t1, L

t′

1 , A1, . . . , Pn, Ltn, Lt′

n , An.Given the four variables of the abstract map, we define its joint distribution

with the following decomposition:

P (P Lt Lt′ A) = P (Lt)P (Lt′)P (A | Lt Lt′)

n∏

i=1

P (Pi Lti Lt′

i Ai | Lt). (2)

In this decomposition, P (Lt) and P (Lt′) are defined as uniform distributions.All the terms of the form P (Pi Lt

i Lt′

i Ai | [Lt = c]) are defined as follows:when c 6= ci, the probabilistic dependency between the variables Pi, Lt

i, Lt′

i ,Ai of the map ci is supposed unknown, therefore defined by a uniform dis-tribution. Whereas when c = ci, this dependency is exactly what the map ci

defines. Therefore this term is a question to the description ci, but a questionthat includes the whole sub-description by asking for the joint distribution itdefines. Since the last term, P (A | Lt Lt′), only includes symbolic variablesthat have a small number of values, it makes sense to define it as a table,which can be easily a priori programmed or learned experimentally.

The abstract Bayesian Map is now fully defined, and, given n underlyingmaps, can be automatically built. The last step is to verify that it generatesuseful behaviors. We will examine the guide questions of localization, predic-tion and control.

The localization question leads to the following inference (derivation omit-ted): P (Lt | P ) ∝∏n

i=1 P (Pi Lti Lt′

i Ai | Lt). The interpretation of this resultwill be explained with an example, Section 6. The derivations for solving theprediction P (Lt′ | A Lt) and control P (A | Lt Lt′) questions are also straight-forward, and given Fig. 5.

Recall that the final goal of any Bayesian Map is to provide behaviors.In the abstract map, this is done by answering a question like P (A | [Lt′ =ci] [P = p]): what is the probability distribution over lower level behaviors,

Bayesian Maps 249

knowing all values p of the variables of the lower level, and knowing that wewant to “go to map ci?” Answering this question thus allows selecting the mostrelevant underlying behavior to reach a given high level goal. The computationis as follows:

P (A | Lt′ P ) =1

Z

∑

Lt

(n∏

i=1

P (Pi Lti Lt′

i Ai | Lt)

)P (A | Lt Lt′). (3)

This computation includes the localization question, to weigh the probabilitiesgiven by the control model P (A | Lt Lt′). In other words, the distribution overthe action variable A includes all localization uncertainties. Each underlyingmodel is used, even when the robot is located at a physical location that thismodel is not made for. As a direct consequence, there is no need to decidewhat map the robot is in, or to switch from map to map: the computationconsiders all possibilities and weighs them according to their (localization)probabilities. Therefore the underlying maps need not be “mutually exclusive”in a geographical sense.

6 Experimental validation

We report here an experiment made on the well-known Koala mobile robotplatform (K-team company). In order to keep as much control as possibleover our experiments and the different effects we observe, we simplify thesensorimotor system and its environment. We only use the 16 proximetersPx = Px0 ∧ . . . ∧ Px15 of our robot, and keep two degrees of freedom ofmotor control, via the rotation and translation speed V rot and V trans. Theenvironment we use is a 5 m × 5 m area made of movable planks (see atypical configuration we use Fig. 6). The goal of this experiment is to solve anavigation task: we want the robot to be able to go hide in any corner, as ifthe empty space in the middle of the area were dangerous.

The first programming step is to analyze this task into sub-tasks. Weparticularize three situations that are relevant for solving the task: the robotcan either be near a wall, and it should follow it in order to reach the nearestcorner, or the robot can be in a corner, and it should stop, or finally it couldbe in empty space, and should therefore go straight, so as to leave the exposedarea as quickly as possible.

6.1 Low level Bayesian Maps

Given this analysis, the second programming step is to define one BayesianMap for each of the three situations. They all use the same perception variableP = Px and the same action variable A = V rot ∧ V trans.

The first map, cwall, describes how to navigate in presence of a sin-gle wall, using a location variable Lt = θ ∧ Dist: the phenomenon “wall”


is summed up by an angle θ and a distance Dist. Therefore, cwall definesP (Px θt Distt θt′ Distt

′

V rot V trans | cwall). We have implemented thismap using 12 possible angle values, and 3 different distances. This lead to acompact model, yet accurate enough to solve the sub-tasks we wanted to solve.The dependency structure we choose is (cwall on right hand sides omitted):

P (Px θt Distt θt′ Distt′

V rot V trans)

= P (θt Distt)∏

i

P (Pxi | θt Distt)P (θt′ Distt′

)

P (V rot | θt Distt θt′ Distt′

)P (V trans | θt Distt θt′ Distt′

).

P (θt Distt) and P (θt′ Distt′

) are uniform probability distributions. Each termof the form P (Pxi | θt Distt) is a set of Gaussians, that were identified experi-mentally, by a supervised learning phase: we physically put the robot in all 36possible situations, with respect to the wall, and recorded proximeter values soas to compute experimental means and standard deviations. Finally, the twocontrol terms P (V rot | θt Distt θt′ Distt

′

) and P (V trans | θt Distt θt′ Distt′

)were programmed “by hand”: given the current angle and distance, and theangle and distance to be reached, what should be the motor commands?

This map successfully solves navigation tasks like “follow-wall-right”, “follow-wall-left”, “go-away-from-wall”, “stop”, using behaviors of the same name. Forexample, “follow-wall-right” is defined by the probabilistic question P (V rot V trans | Px [Lt′ =〈90, 1〉]): compute the probability distribution on motor variables knowing thesensory input and knowing that the location to reach is θ = 90 ,̊ Dist = 1(wall on the right at medium distance).

This map is an instance where a Kalman Filter based Bayesian Map couldhave been used instead: for example, if we had required more accuracy on theangle and distance to the wall, using continuous variables. The coarse grainedset of values we used were actually sufficient for our experiments.

The two other Bayesian Maps we define are the following. 1) ccorner de-scribes how to navigate in a corner, using a symbolic location variable thatcan take 4 values: FrontLeft, FrontRight, RearLeft and RearRight. This isenough for solving tasks like “quit-corner-and-follow-right”, “away-from-both-walls”, “stop”. 2) cempty−spacedescribes how to navigate in empty space, i.e.when the sensors do not see anything. The behaviors defined here are “straight-ahead” and “stop”. For brevity, these two Bayesian Maps are not described fur-ther here, the interested reader can find details in Diard’s PhD thesis Diard[2003].

6.2 Abstract Bayesian Map

Given these three maps, the third and final programming step is to apply theabstraction operator on them. We obtain a map c, whose location variableis Lt = {cwall, ccorner, cempty−space}. The action variable lists the behaviors

Bayesian Maps 251

defined by the low level maps: A = {follow-wall-right, go-away-from-wall, . . .}.The rest of the abstract map is according to the schema of Fig. 5.

We want here to discuss the localization question. Let us assume that therobot is in empty space: all its sensors read 0. Let us also assume that therobot is currently applying the “straight-ahead” behavior, that sets V rot andV trans near 0 (no rotation) and 40 (fast forward movement), respectively,using sharp Gaussian distributions.

Let us consider the probability to be in location cempty−space (with wstanding for wall, c for corner and e for empty − space):

P ([Lt = cempty−space] | P )

∝

P (Pw Ltw Lt′

w Aw | [Lt = cempty−space])

P (Pc Ltc Lt′

c Ac | [Lt = cempty−space])

P (Pe Lte Lt′

e Ae | [Lt = cempty−space])

.

Of the three terms of the product, two are uniforms, and one is the jointdistribution given by cempty−space. That joint distribution gives a very highprobability for the current situation, as describing the phenomenon “goingstraight ahead in empty space” basically amounts to favoring sensory readingsof 0 and motor commands near 0 and 40 for V rot and V trans, respectively.The situation is quite the opposite for P ([Lt = cwall] | P ): for example, cwall

does not favor at all this sensory situation. Indeed, the phenomenon “I am neara wall” is closely related to the fact that the sensors actually sense something.The probability of seeing nothing on the sensors knowing that the robot isnear a wall is very low: P ([Lt = cwall] | P ) will be very low. The reasoning issimilar for P ([Lt = ccorner] | P ).

This computation can thus be interpreted as the recognition of the mostpertinent underlying map for a given sensorimotor situation. Alternatively, itcan be seen as a measure of the coherence of the values of the variables ofeach underlying map, or even as a Bayesian comparison of the relevance ofmodels, as assessed by the numerical value of the joint distributions of eachlower level model. Since these distributions include (lower level) location andaction variables, the maps are not only recognized by sensory patterns, butalso by what the robot is currently doing.

The localization question can therefore be used to assess the “validityzones” of the underlying maps, i.e. the places of the environment where thehypotheses of each model hold. Experimentally, we have the robot navigatein the environment, and ask at each time step the localization question. Wecan summarize visually the answer, for example by drawing values for Lt, andreport the drawn value on a Cartesian map of the environment. A (simplifiedbut readable) result is shown Fig. 6. As can be seen, the robot correctly rec-ognizes each situation that it has a model for. Let us note that the resultingzones are not contiguous in the environment: for example, all the corners ofthe environment are associated with the same symbol, namely, ccorner. Thiseffect is known as perceptual aliasing. But this very simple representation is


Fig. 6. 2D projection of the estimated “validity zones” of the maps cwall, ccorner

et cempty−space. The bottom part of the figure is a screenshot of the localizationmodule of the abstract map: it shows the “comparison” and competition betweenthe underlying models. The winner is marked by the central dot: in this case, therobot was near a wall.

sufficient for solving the task that was given to the robot: we report here thatthe behavior “go-hide-in-any-corner” is indeed generated by the abstract map.

Using the abstract Bayesian Map we thus programed, the robot can solvethe task of reaching corners. A typical trajectory for the robot, starting fromthe middle of the arena, is to start by going straight ahead. As soon as acouple of forward sensors sense something, the “empty-space” situation is notrelevant anymore, and the robot applies the best model it has, depending onthe correlation between what the sensors see: if it looks like a wall and moveslike a wall, then the probability for the “wall” model is high; on the other hand,if it rather feels like a corner, then the corner model wins the probabilisticcompetition. Suppose it was near a wall, then it starts to follow it, until acorner is reached. In our first version, the corner model was designed “tooindependently” of the wall model: the validity zone of the ccorner map wastoo small, and seldom visited by the robot as it passed the corner using the“follow-wall-right” behavior, defined by cwall. The robot would then miss thefirst corner, and stop at another one. This shows that the decomposition ofthe task gives independent sub-tasks only as a first approximation. We solvedthe problem by modifying the “corner” model, so that it would recognize acorner on a typical “follow-wall-right” trajectory.

Bayesian Maps 253

7 Conclusion

We have presented the Bayesian Map formalism: it is a generalization of mostprobabilistic models of space found in the literature. Indeed, it drops the usualconstraints on the choice of decomposition, forms, or implementation of theprobability distributions. We have also presented the Abstraction operator,for building hierarchies of Bayesian Maps.

The experiments we presented are of course to be regarded only as “proofsof concept”. Their simplicity also served didactic purposes. However, theseexperiments, in our view, are a successful preliminary step toward applyingour formalism. Part of the current work is of course aimed at enriching theseexperiments, in particular with respect to the scaling up capacity of the for-malism.

Moreover, since each map of the hierarchy is a full probabilistic model itis potentially very rich. Possible computations based on these maps includequestions like the prediction question P (Lt′ | A Lt), which can form thebasis of planning processes. Hierarchies of Bayesian Maps are therefore to beplaced alongside model based approaches, instead of pure reactive approaches.Exploiting such knowledge by integrating a planning process in our BayesianMap formalism is also part of the ongoing work.

Acknowledgements

This work has been supported by the BIBA european project (IST-2001-32115). The authors wish to thank Emmanuel Mazer, Alain Berthoz andPanagiota Panagiotaki for many discussions and comments on this work.

References

H. Attias. Planning by probabilistic inference. In Ninth International Work-shop on Artificial Intelligence and Statistics Proceedings, 2003.

A. Berthoz. The Brain’s Sense of Movement. Harvard University Press, Cam-bridge, MA, 2000.

P. Bessière, E. Dedieu, O. Lebeltel, E. Mazer, and K. Mekhnacha. Inter-prétation vs. description I˜ : Proposition pour une théorie probabiliste dessystèmes cognitifs sensori-moteurs. Intellectica, 26-27:257–311, 1998.

C. Boutilier, T. Dean, and S. Hanks. Decision theoretic planning: Structuralassumptions and computational leverage. Journal of Artificial IntelligenceResearch, 10:1–94, 1999.

W. Burgard, A. B. Cremers, D. Fox, D. Hähnel, G. Lakemeyer, D. Schultz,W. Steiner, and S. Thrun. Experiences with an interactive museum tour-guide robot. Artificial Intelligence, 114(1–2):3–55, 1999.


J. Diard. La carte bayésienne – Un modèle probabiliste hiérarchique pourla navigation en robotique mobile. Thèse de doctorat, Institut NationalPolytechnique de Grenoble, Grenoble, France, Janvier 2003.

J. Diard, P. Bessière, and E. Mazer. A survey of probabilistic models, usingthe bayesian programming methodology as a unifying framework. In TheSecond Int. Conf. on Computational Intelligence, Robotics and AutonomousSystems (CIRAS), Singapore, December 2003.

J. Diard, P. Bessière, and E. Mazer. Merging probabilistic models of naviga-tion: the bayesian map and the superposition operator. In Proceedings ofthe IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS05), pages 668–673, 2005.

D. Fox, W. Burgard, F. Dellaert, and S. Thrun. Monte carlo localization˜ :Efficient position estimation for mobile robots. In Proceedings of the AAAINational Conference on Artificial Intelligence, Orlando, FL, 1999.

M. Franz and H. Mallot. Biomimetic robot navigation. Robotics and Au-tonomous Systems, 30:133–153, 2000.

E. Gilet, J. Diard, and P. Bessière. Incremental learning of bayesian sensori-motor models: from low-level behaviors to the large-scale structure of theenvironment. In International Conference Spatial Cognition 06 (submittedto), 2006.

K. Gothard, W. Skaggs, K. Moore, and B. McNaughton. Binding of hippocam-pal CA1 neural activity to multiple reference frames in a landmark-basednavigation task. Journal of Neuroscience, 16(2):823–835, 1996.

T. Hartley and N. Burgess. Encyclopedia of Cognitive Science (in press),chapter Models of spatial cognition, page 369. Macmillan, 2002.

M. Hauskrecht, N. Meuleau, L. P. Kaelbling, T. Dean, and C. Boutilier. Hi-erarchical solution of Markov decision processes using macro-actions. InG. F. Cooper and S. Moral, editors, Proceedings of the 14th Conf. on Un-certainty in Artificial Intelligence (UAI-98), pages 220–229, San Francisco,July, 24–26 1998. Morgan Kaufmann. ISBN 1-55860-555-X.

L. F. Jacobs. The evolution of the cognitive map. Brain, behavior and evolu-tion, 62:128–139, 2003.

L. F. Jacobs and F. Schenk. Unpacking the cognitive map: the parallelmap theory of hippocampal function. Psychological Review, 110(2):285–315, 2003.

E. T. Jaynes. Probability Theory: The Logic of Science. Cambridge UniversityPress, June 2003.

L. Kaelbling, M. Littman, and A. Cassandra. Planning and acting in par-tially observable stochastic domains. Artificial Intelligence, 101(1-2):99–134, 1998.

L. Kavraki, P. Svestka, J.-C. Latombe, and M. Overmars. Probabilisticroadmaps for path planning in high-dimensional configuration spaces. IEEETrans. on Robotics and Automation, 12(4):566–580, 1996.

B. J. Kuipers. The spatial semantic hierarchy. Artificial Intelligence, 119(1–2):191–233, 2000.

Bayesian Maps 255

B. J. Kuipers. A hierarchy of qualitative representations for space. In WorkingPapers of the Tenth International Workshop on Qualitative Reasoning (QR-96), 1996.

D. Lambrinos, R. Möller, T. Labhart, R. Pfeifer, and R. Wehner. A mobilerobot employing insect strategies for navigation. Robotics and AutonomousSystems (Special issue on Biomimetic Robotics), 30:39–64, 2000.

T. Lane and L. P. Kaelbling. Toward hierarchical decomposition for planningin uncertain environments. In Proceedings of the 2001 IJCAI Workshopon Planning under Uncertainty and Incomplete Information, Seattle, WA,August 2001. AAAI Press.

T. Lane and L. P. Kaelbling. Nearly deterministic abstractions of markovdecision processes. In 18th Nat. Conf. on Artificial Intelligence, 2002.

J.-C. Latombe. Robot Motion Planning. Kluwer Academic Publishers, Boston,1991. ISBN 0792391292.

O. Lebeltel, P. Bessière, J. Diard, and E. Mazer. Bayesian robot programming.Autonomous Robots (in press), 16(1), 2004.

J. Leonard, H. Durrant-Whyte, and I. Cox. Dynamic map-building for anautonomous mobile robot. The International Journal of Robotics Research,11(4):286–298, 1992.

J. J. Leonard and H. F. Durrant-Whyte. Mobile robot localization by trackinggeometric beacons. IEEE Transactions on Robotics and Automation, 7(3):376–382, June 1991.

T. S. Levitt and D. T. Lawton. Qualitative navigation for mobile robots.Artificial Intelligence, 44(3):305–360, 1990.

E. Mazer, J.-M. Ahuactzin, and P. Bessière. The ariadne’s clew algorithm.Journal of Artificial Intelligence Research (JAIR), 9:295–31, 1998.

K. Murphy. Dynamic Bayesian Networks: Representation, Inference andLearning. Ph.D. thesis, University of California, Berkeley, Berkeley, CA,July 2002.

J. Pineau and S. Thrun. An integrated approach to hierarchy and abstrac-tion for POMDPs. Technical Report CMU-RI-TR-02-21, Carnegie MellonUniversity, August 2002.

L. R. Rabiner and B.-H. Juang. Fundamentals of Speech Recognition, chap-ter Theory and implementation of Hidden Markov Models, pages 321–389.Prentice Hall, Englewood Cliffs, New Jersey, 1993. ISBN 0-13-015157-2.

A. D. Redish and D. S. Touretzky. Cognitive maps beyond the hippocampus.Hippocampus, 7(1):15–35, 1997.

S. Roweis and Z. Ghahramani. A unifying review of linear gaussian models.Neural Computation, 11(2):305–345, February 1999.

É. Simonin, J. Diard, and P. Bessière. Learning Bayesian models of senso-rimotor interaction: from random exploration toward the discovery of newbehaviors. In Proceedings of the IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS05), pages 1226–1231, 2005.


P. Smyth, D. Heckerman, and M. I. Jordan. Probabilistic independence net-works for hidden markov probability models. Neural Computation, 9(2):227–269, 1997.

R. Stackman and A. Herbert. Rats with lesions of the vestibular system re-quire a visual landmark for spatial navigation. Behavioural Brain Research,128:27–40, 2002.

R. Stackman, A. Clark, and J. Taube. Hippocampal representations requirevestibular input. Hippocampus, 12:291–303, 2002.

P. Svestka and M. Overmars. Probabilistic path planning. In J.-P. Laumond,editor, Lecture Notes in Control and Information Sciences 229. Springer,1998. ISBN 3-540-76219-1.

S. Thrun. Probabilistic algorithms in robotics. AI Magazine, 21(4):93–109,2000.

S. Thrun. Robotic mapping: A survey. Technical Report CMU-CS-02-111,Carnegie Mellon University, February 2002.

S. Thrun. Learning metric-topological maps for indoor mobile robot naviga-tion. Artificial Intelligence, 99(1):21–71, 1998.

S. Thrun, A. Bücken, W. Burgard, D. Fox, T. Fröhlinghaus, D. Hennig, T. Hof-mann, M. Krell, and T. Schmidt. Map learning and high-speed navigation inRHINO. In D. Kortenkamp, R. Bonasso, and R. Murphy, editors, AI-basedMobile Robots˜ : Case Studies of Successful Robot Systems, pages 580–586.MIT Press, 1998.

S. Thrun, M. Bennewitz, W. Burgard, A. B. Cremers, F. Dellaert, D. Fox,D. Hähnel, C. Rosenberg, N. Roy, J. Schulte, and D. Schulz. MINERVA˜ :A second generation mobile tour-guide robot. In Proceedings of the IEEEInternational Conference on Robotics and Automation (ICRA), 1999a.

S. Thrun, M. Bennewitz, W. Burgard, A. B. Cremers, F. Dellaert, D. Fox,D. Hähnel, C. Rosenberg, N. Roy, J. Schulte, and D. Schulz. MINERVA˜ :A tour-guide robot that learns. In W. Burgard, T. Christaller, and A. B.Cremers, editors, Proceedings of the 23rd Annual German Conference onAdvances in Artificial Intelligence (KI-99), volume 1701 of LNAI, pages14–26, Berlin, September,˜ 13–15 1999b. Springer. ISBN 3-540-66495-5.

E. Tolman. Cognitive maps in rats and men. Psychol. Rev., 55:189–208, 1948.N. Tomatis, I. Nourbakhsh, K. Arras, and R. Siegwart. A hybrid approach

for robust and precise mobile robot navigation with compact environmentmodeling. In Proceedings of the 2001 IEEE International Conference onRobotics and Automation (ICRA’01), pages 1111–1116, 2001.

N. Tomatis, I. Nourbakhsh, and R. Siegwart. Hybrid simultaneous localizationand map building: a natural integration of topological and metric. Roboticsand Autonomous Systems, 44:3–14, 2003.

D. S. Touretzky and A. D. Redish. A theory of rodent navigation based oninteracting representations of space. Hippocampus, 6:247–270, 1996.

O. Trullier, S. Wiener, A. Berthoz, and J.-A. Meyer. Biologically-based arti-ficial navigation systems: Review and prospects. Progress in Neurobiology,51:483–544, 1997.

Bayesian Maps 257

A. C. Victorino and P. Rives. An hybrid representation well-adapted to theexploration of large scale indoors environments. In Proceedings of the IEEEInternational Conference on Robotics and Automation (ICRA04), pages2930–2935, New Orleans, LA, USA, 2004.

R. F. Wang and E. S. Spelke. Updating egocentric representations in humannavigation. Cognition, pages 215–250, 2000.

R. F. Wang and E. S. Spelke. Human spatial representation: insights fromanimals. TRENDS in Cognitive Science, 6(9):376–382, 2002.

Documents

Bayesian Maps: probabilistic and hierarchical models for ... · tions, they usually have no trouble navigating from the sofa to the bed, or learning to do so when moving in a new