Int J of Soc Robotics (2016) 8:331–351DOI 10.1007/s12369-016-0342-2

Understanding Human Avoidance Behavior: Interaction-AwareDecision Making Based on Game Theory

Annemarie Turnwald1 · Daniel Althoff2 · Dirk Wollherr1 · Martin Buss1

Accepted: 4 February 2016 / Published online: 19 February 2016© The Author(s) 2016. This article is published with open access at Springerlink.com

Abstract Being aware of mutual influences between indi-viduals is a major requirement a robot to efficiently operatein human populated environments. This is especially truefor the navigation among humans with its mutual avoid-ance maneuvers. While humans easily manage this task,robotic systems are still facing problems. Most of the recentapproaches concentrate on predicting the motions of humansindividually and deciding afterwards. Thereby, interactivityis mostly neglected. In this work, we go one step back andfocus on understanding the underlying principle of humandecision making in the presence of multiple humans. Non-cooperative game theory is applied to formulate the problemof predicting the decisions of multiple humans that interactwhich each other during navigation. Therefore, we use thetheory of Nash equilibria in static and dynamic games wheredifferent cost functions from literature rate the payoffs of theindividual humans. The approach anticipates collisions andadditionally reasons about several avoidance maneuvers ofall humans. For the evaluation of the game theoretic approachwe recorded trajectories of humans passing each other. Theevaluation shows that game theory is able to reproduce the

B Annemarie Turnwalda.turnwald@tum.de

Daniel Althoffalthoff@andrew.cmu.edu

Dirk Wollherrdw@tum.de

Martin Bussmb@tum.de

1 Chair of Automatic Control Engineering (LSR), TechnischeUniversität München, 80290 Munich, Germany

2 Robotics Institute, Carnegie Mellon University, Pittsburgh,PA 15213, USA

decision process of humans more accurately than a decisionmodel that predicts humans individually.

Keywords Game theory · Interaction-aware decisionmaking · Human motion analysis · Interactivity duringnavigation

1 Introduction

Robots have evolved immensely in the past decades, but thereis still a long way to go in terms of enabling them to per-manently operate in human populated environments. Firstachievements include robots that navigate autonomously tounknown places [7,53] or guide people at fairs [62]. Apartfrom that, autonomous vehicles already navigate in urbanenvironments among human-driven vehicles [81]. Otherrobots interact physically with humans, for example theyhand over objects [72] or assist elderly people [63]. Theseapplications show that the barrier between robots and humanshas been fading, which leads to a major challenge in robot-ics: ensuring reliable and socially accepted motion in orderto realize the human-robot coexistence. A vital factor forachieving this goal is the awareness of the mutual influ-ence between human individuals and the robotic systems.Modern robotic systems have to consider that humans areinteraction-aware: they reason about the impact of possi-ble future actions on the surrounding and expect similaranticipation from everyone else [6,55,61]. Of our particu-lar interest is the interaction-aware navigation of humans,meaning the conditionally cooperative behavior that leadsto mutual avoidance maneuvers. Common motion plannersneglect this notion and focus on independent motion predic-tion of individuals. Thereby, the prediction can be unreliablebecause it is indifferent to humans that might avoid the robot.

123

332 Int J of Soc Robotics (2016) 8:331–351

It is important, that the future trajectory of the humandependson the motion of the surrounding humans and on the motionof the robot. Consequences are unnecessary detours, inef-ficient stop and go motions, or a complete standstill if theplanner fails to identify a collision-free trajectory. The worstcase is a collision with a human. Trautman et al. [78] arguethat these problems would still exist with perfect predic-tion and that they could only be resolved by anticipatingthe human mutual collision avoidance. This clarifies that theindividual motion prediction of humans and a consecutiveplanning of the robot motion is a poor model for human deci-sion making. Hence, a main reason of robotic problems liesin an insufficient model of the human decision process. Thatis why we model human decision making in the presence ofmultiple humans during navigation. We formulate a decisionmodel that considers interaction-awareness and reasoning.On top, the model can cope with human diversity: humansdecide individually (decentralized planning) and have differ-ent preferences (e.g., speed, goal).

In this paper, the problem formulation of interaction-aware decisionmaking is based on game theory.Game theoryis “the study of mathematical models of conflict and coop-eration between intelligent rational decision-makers” [51,p. 1]. It extends the traditional optimal control theory to adecentralized multi-agent decision problem [3, p. 3]. Thus,prediction and planning are considered simultaneously. Wespecifically choose to use game theory because it is amathematical formulation and incorporates reasoning aboutpossible actions of others and consequences of interde-pendencies, i.e. interaction-awareness. It further allows forindividual decision making and individual utility functionsthat capture preferences. Its strength lies within its generaliz-ability. Accordingly, a variety ofmodeling approaches exists,as well as diverse solution concepts that aim to predict thedecision of agents, for example which trajectory they willtake. Within this work, our focus lies on the solution con-cept of Nash equilibria in non-cooperative games. These areequilibria where no one gains anything by only changing theown decision.

General Idea: Approximating the decision making ofhumans during interaction-aware navigation with the the-ory of Nash equilibria in non-cooperative games.

In the following, two possible ways to model humannavigation are presented. One model assumes simultaneousdecision making, the other one sequential decision making.Both models are combined with cost functions from liter-ature. We evaluate for which of these combinations, Nash’stheory reproduces the navigational decisions of humans best.Thereby, the evaluation is based on captured human motiondata to ensure real human behavior. Additionally, the gametheoretic approach is compared with a common predictionbased decision model. Our intention is to draw further con-clusions about human navigational behavior and to highlight

Fig. 1 Deciding human-like during navigation; interactivity, likemutual avoidance maneuvers, needs to be considered

the potential of game theory for this problem. Note, thatthe presented work focuses on humans and on modelingtheir decisions. We do not yet present a motion predic-tion or motion planning algorithm for robots. However, thederived knowledge can improvemotion prediction of humansand robot motion planning, as well as the social accep-tance of robots. If the behavior of a robot is based on ahuman-like decision process, its intentions are far easier tointerpret and as a result, a human interaction partner feelsmore secure [12,77] (Fig. 1).

This paper is organized as follows. Sect. 2 surveys thework related to humanmotion analysis and interaction-awarenavigation. The next section gives an outline of the game the-oretic method used to analyze human motion (Sect. 3); twodifferent models and five possible cost functions are pre-sented. The experimental setup and the evaluation methodare discussed in Sect. 4, followed by the results in Sect. 5,and possible extensions in Sect. 6.

2 Related Work

The problem of modeling interaction-awareness of severalagents has been addressed in different areas including humanmotion analysis, computer animation and robot motion plan-ning. It is particularly attractive for a branch within the latterfield—the socially-aware robot navigation [39,66]. Relatedexperiments and motion planners that consider interactivityare presented in this section. Additionally, the section elab-orates about applications of game theory in motion planningand decision making problems.

Various groups of researches have studied human col-lision avoidance during walking [5,6,15,16,31,54,55,61].They have been interested inwhen,where, aswell as towhichextent humans adjust their path or velocity to avoid a collisionwith another dynamic object. All studies agree that humansanticipate the future motion of dynamic objects and possiblecollisions. That means that humans include prediction intotheir own motion planning and do not solely react. How-ever, parts of these studies neglect the interaction-awareness

123

Int J of Soc Robotics (2016) 8:331–351 333

of humans during walking by only considering avoidancemaneuvers with a passive, dynamic object. For example, thesubjects of the studies by Cinelli and Patla [15,16] had toavoid a human-shaped doll that was moving towards them;Basili et al. [5] and Huber et al. [31] asked their participantsto cross paths with a non-reacting human.

In contrast are the studies from Pettré et al. [61], vanBasten et al. [6], and Olivier et al. [54,55]. These authorstold two participants to avoid a collision, which revealedthat humans collaboratively adjust their gait. Interestingly,the amount of adjustment was unequally distributed [55,61]:the person passing first had less effort because s/he mainlyadopted the velocity, whereas the one giving way adjustedboth, velocity and path. In summary, analyzing human loco-motor trajectories shows up the important characteristics ofhuman collision avoidance. This can be used to evaluateor enhance the human-likeness of motion models. Unfortu-nately, it does not reveal how to reproduce human avoidancebehavior in order to use it for motion prediction or motionplanning.

Researchers have often based human motion models onrepulsive forces acting on particles. That has been especiallypopular for crowd simulations: Pelechano et al. [59] or Sudet al. [76] employ the social forces model [28] where theagents are exposed to different repulsive and attractive forcesdepending on their relative distances; Heïgeas et al. [27]define forces in analogy to a spring-damper systemwith vary-ing stiffness and viscosity values; and Treuille et al. [79] use apotential field approach. However, we refrain from elaborat-ing this field deeper becausemostworks are based on reactiveapproaches and neglect that humans include prediction intheirmotion planning.While thismay be appropriate for highdensity crowds, reactive approaches struggle—accordingto [5,31,61]—with creating locally realistic motions in lowor medium density crowds.

Trautman et al. [78] focus on thesemediumdensity crowdsand plan further ahead by relying on Gaussian processes.The authors define the “Freezing Robot Problem”: once theenvironment gets to crowded, the planner rates all possiblemaneuvers as unsafe due to increasing prediction uncertainty.As a result, the robot “freezes” or performs unnecessarydetours. They argue that this problem would still exist,even without uncertainty and with perfect prediction. Itcould be resolved by anticipating the human collaborativecollision avoidance. They developed a non-parametric sta-tistical model based on Gaussian processes that estimatescrowd interaction from data. Thereby, independent Gaussianprocesses are coupled by a repulsive force between theagents. Experiments verified that the interactive algorithmoutperforms a merely reactive one.

An earlier approach was shown by Reynolds [65]. Heuses different steering behaviors to simulate navigatingagents. One of these behaviors—the unaligned collision

avoidance—predicts future collisions based on a constantvelocity assumption and gets the agents to adjust steeringand velocity to avoid state-time space leading to a collision.

In contrast to this rule based method are approachesbased on velocity obstacles [22], like its probabilistic exten-sions [37]. Van den Berg et al. [9] combine a precomputedroadmap with so-called reciprocal velocity obstacles. Thisapproach is updated in [10] to the optimal reciprocal colli-sion avoidance that guarantees collision-free navigation formultiple robots assuming a holonomicmotionmodel. Furtherassumptions are that each robot possesses perfect knowledgeabout the shape, position and velocity of other robots andobjects in the environment. Extensions that incorporate kine-matic and dynamic constraints exist in [1,74].

However, Pettré et al. [61] state that the works in [65]and [9] lack to simulate the large variety of the humanbehavior because they rely on near-constant anticipation-times and on the common knowledge that all agents apply thesame avoidance strategy. Instead, they presented an approachwhich produces more human-like trajectories: they solvepairwise interactions with a geometrical model based on therelative positions and velocities of the agents. It is tunedwith experimental data and analyzed according its validityfor crowds by Ondrej et al. [56]. The authors state to per-form better, in a sense that the travel duration of the agentsis shorter, when compared to [9] or [28].

Shiomi et al. [71] developed a robot that successfullynavigates within a shopping mall. It relies on an extendedsocial force model that includes the time to collision asa parameter to compute an elliptic repulsion field aroundan agent [85]. Mutual collision avoidance is implicitlyintroduced by calibrating the repulsion force with humanavoidance trajectories. A field trial revealed that the robotusing this method was perceived as safer than if it used a timevarying dynamic window approach [70]. Nevertheless, mostof the mentioned methods assume that an agent’s behaviorcan be described by a limited set of rules.

Recently, learningbasedapproaches are becoming increa-singly popular. Lerner et al. [43] extract trajectories fromvideo data to simulate human crowds. They create a databasecontaining example navigation behaviors that are describedby the spatio-temporal relationship between nearby personsand objects. During the simulation the current state of theenvironment is compared with the entries of the database.The most similar entry defines the trajectory of the agent,thus, implicitly creates reactive behavior. This means thatthe variety of the behaviors is limited to the size of thedatabase. Moreover, all individuals need to be controlledglobally.

Luber et al. [46] proposed an unsupervised learningapproach based on clustering observed, pairwise navigationbehaviors into different motion prototypes. These proto-types are defined by the relative distance of two agents over

123

334 Int J of Soc Robotics (2016) 8:331–351

time and their approaching angle. They are used to derive adynamic cost map for a Theta∗ planner that generates a pathat each time step.

Apart from that, many researchers rely on inverse rein-forcement learning techniques. Kuderer et al. [40] learn acertain navigation policy while a robot is tele-operated. Theprinciple of maximum entropy [88] is used to learn theweights of the feature functions. In particular, homotopyclasses (i.e., on which side to pass an object) are consid-ered. Kretzschmar et al. [38] improve this method further;they used a joint mixture distribution that consists on onehand of a discrete distribution over these homotopy classes,and on the other hand of continuous distributions over thetrajectories in each homotopy class. An experiment revealedthat the resulting trajectories are perceived as more human-like than the one produced by their previous method [40] orthe social force model [28].

An alternative is given by Henry et al. [29] who learnhow a simulated agent has to join a pedestrian flow byusing the density and average flow directions as features.Similarly, Kim and Pineau [36] proposed to use the popu-lation density and velocity of the surrounding objects. Theeffect of the different features in [29] and [36] were investi-gate by Vasquez et al. [82] and compared with social forcefeatures [28]. Results showed that the social force featuresperform best when applied specifically for the learned scene,but seem to generalize worst to other scenes. The features in[29] and [36] are more generalizable and manage similarlywell.

A new approach to consider interaction-awareness is tomodel the navigational decision problem with game theory.By providing the language to formulate decision problems,game theory has already found some ways into robotics. Itis used in robust control [4,23,58], for example for landingan aircraft [23]. Also the task planning of a planetary surfacerover runs with game theoretic tools [32].

The use of game theory is also growing within the roboticresearch community, in particular in thefields ofmotion plan-ning and coordination. LaValle and Hutchinson [42] wereamong the first who proposed game theory for the high-levelplanning of multiple robot coordination. Specific applica-tions are a multi-robot search for several targets [48], theshared exploration of structured workspaces like buildingfloors [73], or coalition formation [24]. Closely related tothese coordination tasks is the family of pursuit-evasion prob-lems. For example in [3,49,83], and can be formulated as azero-sum or differential game. Zhang et al. [86] introduceda control policy for a motion planner that enables a robotto avoid static obstacles and to coordinate its motion withother robots. Their policy is based on zero-sum games andassigning priorities to the different robots. Thus, it eludespossiblemutual avoidancemaneuvers by treating robots witha higher priority as static obstacles. The motion of mul-

tiple robots with the same priority are coordinated withinthe work of Roozbehani et al. [67]. They focused on cross-ings and developed cooperative strategies to resolve conflictsamong autonomous vehicles. Recently, Zhu et al. [87] dis-cussed a game theoretic controller synthesis for multi-robotmotion planning. So far, there has been almost no attemptsto connect game theory with models for human motion—with the exception of Hoogendoorn and Bovy [30]. Theyfocus on simulating crowdmovements andgenerated promis-ing results, especially for pedestrian flows, by formulatingthe walking behavior as a differential game. However, theydo not solve the original problem or discuss a commonsolution concept like equilibrium solutions. They eventuallytransform the problem into an independent optimal controlproblem based on interactive cost terms. They specificallypayed attention on reproducing human-like crowd behavior,hence, their simulation based evaluation is qualitative andassesses the macroscopic group behavior.

Our approach is to analyze human motion—in particu-lar, the human avoidance behavior—from a game theoreticperspective. Using game theory provides several advantagesover existing approaches. Compared tomerely reactivemeth-ods, the key factor of game theory is the mutual anticipationof the influence of other agents’ possible motions on oneselfand vice versa; costs (or payoffs) of own actions can dependon decisions of others. Thus, future interactions are predictedand incorporated which corresponds to human behavior.Moreover, individual cost functions can be assigned to eachagent if desired. As a result, agents can behave asymmet-rically which overcomes the restrictions of most of thementioned algorithms with anticipated collisions avoidance.Learning based methods are very promising because of theirinherent usage of real, human motion data. This is at thesame time a drawback because their validity is dependenton the versatility of the their experimental data. In contrast,game theory offers a more general formulation with a vari-ety of extensions and can apply learned cost functions aswell.

In this paper, interaction-aware decision making is for-mulated as a non-cooperative game, whereas a static and adynamic representation is proposed. Navigational decisionsare predicted by calculating Nash equilibria dependent onalternative cost functions form literature.We further evaluatewhich combination of model and cost function approximatesrecorded human navigation experiment best. Note, thatwe donot present an operational motion planner or prediction algo-rithm in this paper. The presented approach is based on ourprevious work [80].We extend our analysis by also regardingthe Nash equilibria of dynamic games and by examining fouradditional cost functions, which mainly perform better thanthe previously used one. Further, a comparison between thegame theoretic decision model and a commonly used predic-tion based one is conducted.

123

Int J of Soc Robotics (2016) 8:331–351 335

3 Analyzing Interaction-Aware Navigation withGame Theory

For a game theoretic formulation of human decision makingduring navigation we consider two game formulations withfive different cost functions each. Then the solution conceptof Nash equilibria is used to predict possible outcomes of thegame. In the following, game theoretic terms and tools arebriefly explained and an illustrative example is given.

3.1 Defining Static and Dynamic Games

The two considered models are finite, non-cooperative, non-zero-sum, perfect information games. However, one game isstatic, the other one is dynamic. In the static case, decisionsare taken simultaneously, in contrast to deciding sequentiallyin dynamic games. The termnon-cooperative refers to a gametheoretic branchwherein all agents aim tominimize their costindividually.1 One speaks of a nonzero-sum game if the sumof each agent’s costs can differ from zero. Keeping that inmind, the components of the static game are defined by:

Definition 1 (Static Game) Finite, static, non-cooperative,nonzero-sum game [44]:

1. Finite set of N players P = {P1, . . . , PN }.2. Finite action setA = A1×· · ·×AN , whereAi is defined

for each player Pi ∈ P . Each a ji ∈ Ai is referred to as

an action of Pi , with j = {1, 2, . . . , Mi } and Mi beingthe number of actions of Pi .

3. Cost function Ji : A1 × A2 × · · · × AN → R ∪ {∞} foreach player Pi ∈ P .

The game is finite if the number of actions is bounded forall players. The subscript i always refers to the addressedplayer. Each player Pi has different actions a

ji ∈ Ai and a

cost function Ji . The superscript j refers to an action, anda ji is the j th action out of Mi actions of player Pi .In a static game, as defined above, the players decide once

and simultaneously. Hence, navigating agents are modeledas if they observe the situation first, and then decide instinc-tively. This assumption of humans using default collisionavoidance strategies is supported by Huber et al. [31].

Other studies [55,61] in turn state that the amount ofshared effort during the avoidance maneuvers is unequal,depending on who is first. This indicates that humansobserve and react, which may be more accurately modeledby considering sequential decisions. Dynamic games model

1 Non-cooperative in contrast to cooperative/coalitional games wherethe focus is set on what groups of agents—rather than individuals—can gain by forming coalitions. In a nutshell, coalitional game theoryanswers two questions: which coalition will form, and how should thatcoalition divide its payoff among its members [44, p. 70].

these situations where decisions during navigation are takensequentially. In this case, the property of perfect informationin games becomes important to get a complete definitionof the game. A game is a perfect information game if eachplayer perfectly knows about the actions of all players thathappened previously. Thus for the dynamic model of naviga-tion, it is assumed that the agents choose consecutively, andan instant after they observe the actions of the agents actingbefore them. It is unclear if the static or dynamic model ismore accurate. Both models are evaluated and compared inthis work.

A mathematical description of a dynamic game is theextensive form. This form emphasizes the sequential deci-sion making. The components are given by:

Definition 2 (Dynamic Game) Finite, perfect information,dynamic, non-cooperative, nonzero-sum game [44]:

1. Finite set of N players P = {P1, . . . , PN }.2. Finite action set A = A1 × · · · × AN .3. Set of terminal nodes Z .4. Cost function Ji : Z → R ∪ {∞} for each Pi ∈ P .5. Set of choice nodes H.6. Action function χ : H → 2A; assigns each choice node

a set of possible actions.7. Player function ρ : H → P; assigns each choice node a

player Pi ∈ P who chooses the action at the node.8. Successor function σ : H × A → H ∪ Z; uniquely

assigns a choice node and an action a subsequent node.

Informally speaking, a dynamic game is a (graph theoretic)tree, in which each node depicts a decision of one of theagents, each edge depicts an action, and each leaf depicts afinal game outcome.

3.2 Solving Games: Strategies and the NashEquilibrium

The definitions introduced before withhold which actionsa player should choose. Game theorists introduced diversesolution concepts that can be interpreted as an advice or usedas prediction of what is likely to happen. One of the mostfamous solution concepts is the Nash equilibrium: it is anallocationwhere no player can reduce the own cost by chang-ing the strategy if the other players stick to their strategies.Thus, a Nash equilibrium is a best response for everyone. Itimplies that agents aim to minimize their own cost. This iscorroborated by existing literature stating that humans exe-cute their motions by following a minimization principle.Accordingly, humans minimize for example the energy con-sumption of their gait [47,75], or the global length of theirpaths [11]. Also psychologists claim that even infants expecta moving agent as having goals which it aims to achieve in

123

336 Int J of Soc Robotics (2016) 8:331–351

a rational way, like by taking the shortest path [19]. Becausethese statements bring the Nash equilibrium into focus, weconcentrate on this solution concept. Nevertheless, anothersolution concept—the Pareto optimality—is discussed andbriefly evaluated in Sect. 6.1.

Before presenting a mathematical definition of a Nashequilibrium, the notion strategy is introduced because it dif-fers for static and dynamic games. Similar for both types isthat each player Pi has to play a strategy s

ji out of the strategy

set Si . This strategy can be pure or mixed. A pure strategy isdeterministic; whenever a pure strategy is played in a game,the same actions are chosen. If a mixed strategy is played, apure strategy is chosen stochasticallywith a fixed probability.Thus, a game may reach different outcomes with the same,mixed strategy. In a static game, for a player Pi choosinga pure strategy s ji ∈ Si is equivalent to choosing an action

a ji ∈ Ai , hence, s

ji = a j

i . Defining a strategy in the dynamiccase is more complex because one has to define for eachchoice node of a player in H which action is to be played,whether or not the choice node is reached during the game.Thus, the pure strategies of a player Pi in a dynamic gameconsist of theCartesian product

∏h∈H,ρ(h)=Pi χ(h) [44]. For

further explanation an example is given in Sect. 3.4.Keeping that in mind, we denote a combination of strate-

gies of each player as an allocation s = (s j1 , . . . , s jN ). ANash

equilibrium s∗ = (s j∗

1 , . . . , s j∗

N ) is marked with an asteriskand is a best response for everyone, assuming possible strate-gies and cost functions are common knowledge. It is definedby:

Definition 3 (Nash equilibrium) The N -tuple of strategies(s j

∗1 , . . . , s j

∗N ), with s j

∗i ∈ Si , constitutes a non-cooperative

Nash equilibrium for a N -player game if the following Ninequalities are satisfied for all s ji ∈ Si :

J1(s j

∗1 , s j

∗2 , . . . , s j

∗N

)≤ J1

(s j1 , s j

∗2 , s j

∗3 . . . , s j

∗N

)

J2(s j

∗1 , s j

∗2 , . . . , s j

∗N

)≤ J2

(s j

∗1 , s j2 , s j

∗3 , . . . , s j

∗N

)

...

JN(s j

∗1 , s j

∗2 , . . . , s j

∗N

)≤ JN

(s j

∗1 , . . . , s j

∗N−1, s

jN

).

(1)

Note that the existence of a Nash equilibrium is guaranteedgiven that mixed strategies are allowed [52]. Choosing a spe-cific form for the cost function Ji even ensures the existenceof a Nash equilibrium in pure strategies (see Sect. 3.3). Pos-sible choices for such a Ji regarding human navigation arediscussed in the next section.

It is further important that theNash equilibrium is boundedto two main assumptions: common knowledge of all players,and strictly rational behavior of all players. Common knowl-edge implies that all players know about the whole action

set and the cost functions. We assume that humans gain their(common) knowledge through experience and their ability totake perspective. Humans learn in their everyday life whatalternatives exist to reach a goal and how other people behavewhile walking. At the same time humans are able to view asituation from another’s point-of-view and infer about theirpossible actions and intentions. Rational behavior is definedas a behavior that maximizes an expected utility [17] (i.e.,minimize expected cost). Section 6.1 discusses rationalitymore detailed, as well as to which extent both assumptionsare justified for interaction-aware decision making duringnavigation. In case either of these assumptions is violated,game theory extensions are proposed.

3.3 Determining Cost Functions for Human Navigation

The mathematical definition of a Nash equilibrium in Defin-ition 3 demonstrates that the accuracy of the game theoreticprediction is dependent onboth the choice of the solution con-cept and the cost function. This section presents five differentchoices to rate the cost for human navigation. The evaluationis presented in Sect. 4. Thereby, each cost function Ji consistsof an independent component J and an interactive compo-nent Ji :

Ji(a j1 , . . . , a

ji , . . . , a j

N

)= J

(a ji

)+ Ji

(a j1 , . . . , a

ji , . . . , a j

N

).

(2)

Note that this partitioning clarifies that the game theoreticformulation results in an independent set of optimal controlproblems if no interaction occurs. J is only dependent on theaction a j

i that player Pi considers. It rates for example thelength or time of the trajectory. The interactive component Jinamely contains the interactive cost. It is not only dependenton the own choice of action but also on the other players’actions.

Four of the considered cost functions (I–IV) assume thathumans prefer trajectories that are, above all, without colli-sion and otherwise minimize their cost with respect to theirfree-space motion. The fifth cost function (V) contains anadditional cost term that rates how close humans pass eachother, hence, shares some characteristics with the social forcemodel. This cost function will be discussed last. For the otherfour cost functions a common interactive component Ji canbe defined.

Cost Function I–IV The cost functions consider a collisionto be the only possible interaction.

J I−IVi

(a j1 , . . . , a

jN

):=

{∞ if at least one collision occurs,

0 else.

(3)

123

Int J of Soc Robotics (2016) 8:331–351 337

J I−IVi becomes infinity in case that action a j

i leads to a colli-sion with the strategy of another player, otherwise the term iszero. An action corresponds to a discrete trajectory with thestates x(t), with a j

i = (x(1), x(2), . . . , x(T )). Two trajecto-ries collide if the Euclidean distance between two positionsis smaller than a threshold R at any time t .

By choosing a cost function in the form of Eq. (2) withEq. (3), the existence of a Nash equilibrium in pure strategiesis guaranteed: a player’s cost is either J (a j

i ) or infinity. If it isinfinity for a special allocation, all other players with whoma collision would occur also have infinite cost. But this is nota best response for any player (only in case all actions of aplayer would result in a collision).

In the following, the choices for the independent compo-nent J of the first four cost functions are presented. Two ofthem need a trajectory as input, the other ones merely needthe path information.

Cost Function I A frequently used cost function in motionplanning [41] is the length L of the path.

J I(a ji

):= L

(a ji

). (4)

Another cost function using path input is given byPapadopoulos et al. [57]. They learned the parameters of acost function by studying the geometry of the path in free-space andbyusing inverse optimal control. Theirmodel baseson non-holonomic motions along a path that is approximatedby line segments with the state vector x(k) = [x, y, ϕ]T atthe kth segment of the path. x and y denote positions andϕ the orientation. Their cost function depends only on theshape of the path and is invariant to changes of the velocity.

x(k + 1) = x(k) + λ(k) cos(ϕ(k))

y(k + 1) = y(k) + λ(k) sin(ϕ(k))

ϕ(k + 1) = ϕ(k) + λ(k)κ(k),

(5)

with κ being the curvature and λ(k) being the length of thekth segment. Let K be the total number of segments. Then apossible cost function is:

Cost Function II The cost function is based on Eq. (5) andaccounts for the energy related to the curvature, and for thedistance between the current state and the goal state.

J II(a ji

):= 1

2

K−1∑

t=0

λ(k)(κ(k))2(1 + cTΔx2(k)), (6)

with Δx2 = [(x(k) − x(K ))2, (y(k) − y(K ))2, (ϕ(k) −ϕ(K ))2]T and cT = [125, 42, 190]. The distances from thecurrent state to the goal state can be interpreted as space-varying weights on the curvature [57].

Both mentioned cost functions are path and thus notvelocity dependent. Another possibility is to use trajectoryinformation. Consequently, we also consider the followingcost functions.

Cost Function IIIThe function rates thedurationT neededfor the player Pi to play an action, meaning to walk alongthe trajectory.

J III(a ji

):= T

(a ji

). (7)

A more complex cost function is given by Mombauret al. [50] who studied the run of human locomotor tra-jectories during goal-directed walking in free-space. Theystate that human trajectories are optimized according to anunderlying principle which was learned with inverse opti-mal control. In contrast to cost function J II, they assumethe motion model to be holonomic with the state vectorx(t) = [x, y, ϕ, vforw, vang, vorth]T and the control vectoru(t) = [uforw, uang, uorth]T (forward, angular, orthogonalacceleration).

Cost Function IV The cost function assigns a cost for theexecution time T needed (as in Eq. (7)). Additionally, itfavors sparse accelerations and the human to be orientedtowards the goal.

J IV := T (a ji ) +

T(a ji

)

∑

t=0

cTu(t)2, (8)

with u(t)2 = [u2forw, u2ang, u2orth, ψ

2]Tt , and ψ being the dif-ference between the angular difference to the goal denotedas z = [x(T ), y(T )]T and the human body orientation ϕ;

ψ(x(t), z) = arctan(y(T )−y(t)x(T )−x(t)

)− ϕ(t).

The parameter vector is cT = [1.2, 1.7, 0.7, 5.2] [50].The last cost function is adopted in large parts from Pel-

legrini et al. [60]. They showed that a tracking algorithmperforms better if it takes social interactions between pedes-trians into account as well as their orientation towards a goal.Minimizing a learned cost function allowed for calculatingthe next expected velocity of the tracked object. This costfunction is used here to rate a whole trajectory. A holonomicmotion model with the state vector x(t) = [x, y]T and thecontrol vector u(t) = [vx , vy]T is chosen.

Cost Function VThe cost function rates if a trajectory leadstowards its goal andmaintains a desired speed. Additionally,it rewards trajectories that steer an agent away from anexpected point of closest approach to another agent.

123

338 Int J of Soc Robotics (2016) 8:331–351

JV(a ji ) :=

T (a ji )∑

t=0

c1G(t) + c2V(t) (9)

JVi (a j1 , . . . , a

jN ) :=

⎧⎪⎪⎨

⎪⎪⎩

∞ if collision,T (a j

i )∑

t=0

N∑

n �=iwn(t)Iin(t) else.

(10)

G(t) is dependent on the goal z = [x(T ), y(T )]T , and V(t)depends on a desired speed vdes, with

G(t) = − (z − x(t))Tu(t)

‖z − x(t)‖‖u(t)‖ , V(t) = (vdes − ‖u(t)‖)2.

Each fellow player of Pi gets assigned a weight wn(t) deter-mined by distances and angular displacements of players toeach other.2 The additional interactive cost resulting fromplayer Pi approaching player Pn is denoted by:

Iin(t) = exp(−d2in(t)

2σ 2

),

where d2in(t) is the square distance between the positionsof Pi and Pn at the expected point of closest approach.The calculation of that point is based on a constant veloc-ity assumption. It is similar to the social force model, butdiffers in a crucial way: instead of modeling humans at theircurrent positions, the expected point of closest approach ispredicted andused as origin of the repulsion.This implies thathumans include prediction into the motion planning, ratherthan being reactive particles [60]. The parameter vector iscT = [c1, c2, σ ] = [2.073, 2.330, 0.361] [60].

3.4 Formulating Interaction-Aware Decision Makingwith Game Theory

In this section, the game theoretic tools described aboveare applied to a navigational decision problem. A static anddynamic game is set up and their Nash equilibria are calcu-lated. Thereby, a mapping between game theoretic terms andnavigational components is given with the aid of an exam-ple illustrating the navigation on a pavement. The example isdepicted in Fig. 2 where two agents want to pass each other.

First, we map the components of a static game in Def-inition 1 to the pavement scenario. The two agents are theplayers P1 and P2. Choosing an action a j

i is equivalent tochoosing a trajectory. In the example inFig. 2, eachplayer canchoose one out of five trajectories, thus the action set of Pi isAi = {a1i , a2i , . . . , a5i }. In the static case, thismirrors directlythe set of pure strategies Si = {s1i , . . . , s5i } = {a1i , . . . , a5i }.Figure 3 assigns each action a trajectory and shows the cost

2 For a detailed calculation please refer to [60].

Fig. 2 Example for interaction-aware navigation of humans on a pave-ment. Interaction may be a mutual avoidance maneuver. The situationis modeled as a static game in Table 1 and Fig. 3 and as a dynamic gamein Fig. 4

Fig. 3 Illustration of Fig. 2 as static game. The actions of each playerand the cost of the trajectories are shown (assuming it is collision-free)

Table 1 Static game

P1\P2 a12 a22 a32 a42 a52

a11 5|5 5|4 5|1 ∞ ∞a21 4|5 4|4 ∞ ∞ ∞a31 1|5 ∞ ∞ ∞ 1|3a41 ∞ ∞ ∞ 2|2 2|3a51 ∞ ∞ 3|1 3|2 3|3

The cells depict cost pairs J1|J2 dependent on actions a ji . Actions and

corresponding cost are shown in Fig. 3. In case of a collision the costis infinity. Nash equilibria are circled

component J , that choosing a trajectory entails (assumingit is collision-free). The cost is chosen such that it is pro-portional to the length of the path and that passing rightis favorable to passing left. The cost Ji (s

j1 , s j2 ) of a player

depending on the allocation are written down in a matrix asshown in Table 1, where each cell contains a cost pair J1|J2.

After all components aremapped, the pureNash equilibriaof the game are computed. In the static, two-player case theinequalities in Eq. (1) reduce to:

J1(s j∗1 , s j∗2

)= min

{J1

(s j1 , s j∗2

)}∀s j1 ∈ S1,

J2(s j∗1 , s j∗2

)= min

{J2

(s j∗1 , s j2

)}∀s j2 ∈ S2.

(11)

Informally speaking, a cell in Table 1 (i.e., an allocation)is a pure Nash equilibrium if a) the cost entry J1 is less orequal than all other costs J1 in its column and b) the cost

123

Int J of Soc Robotics (2016) 8:331–351 339

Fig. 4 Dynamic representation of the pavement example in Fig. 2.The upper part shows the actions of the players with its correspondingcost without collision. The lower part is the tree representation of thedynamic game with the cost pairs J1|J2 at the leafs (the first entryrefers to P1, the second to P2, respectively). The subgame-perfect Nashequilibrium is circled

entry J2 is less or equal than all other costs J2 in its row.Four allocations satisfy both conditions. They are circled inTable 1. For example, the allocation s∗ = (s3∗1 , s5∗2 ) is aNash equilibrium. Choosing this equilibrium means that theplayers decide simultaneously to play trajectory a31 and a52 ,respectively.

In case that the agents decide sequentially, the dynamicgame is used. Definition 2 is applied on the pavementsituation in Fig. 2. The dynamic game is depicted as atree. However, its illustration gets confusing with too manybranch-offs. For that reason the example is altered by onlyregarding three possible trajectories for each agent as shownin Fig. 4. This leads to the action set A = A1 × A2 ={aL1 , aM1 , aR

1 , aL2 , aM2 , aR2 }. Similar to the static game, we

have two players P1 and P2. However, a player function ρ isneeded that states the agent acting first. For this example P1is chosen. After observing one out of three actions of P1, P2reacts by playing one of his actions in turn. This results in thetree shown in Fig. 4. The terminal nodes—the leafs—assignthe cost for each player as defined by the cost function Ji .The cost functions are the same as in the static game.

One major difference between static and dynamic gameslies in their strategy space. P1 has one choice node andthree actions, thus 31 = 3 different strategies with withS1 = {s11 , s21 , s31} = {aL1 , aM1 , aR

1 }. More interesting, playerP2 has three choice nodes, thus already 33 = 27 strategies,S2 = {s12 , . . . , s272 } = {(aL2 , aL2 , aL2 ), (aR

2 , aL2 , aL2 ), . . . }.The number of strategies is that highbecause onehas to definea choice of action for each choice node. The reason why thisrefinement is necessary is that a Nash equilibrium was orig-inally defined for static games. For the definition in Eq. (1)to be still valid for the dynamic game, one has to define thestrategies such that they state for each choice node of a player

which action is to be played—whether or not the choicenode is reached during the game [44]. However, this defi-nition allows for unlikely equilibrium solutions in a dynamicgame. For example, one of theNash equilibria in the dynamiccase is the allocation s∗ = (aL∗

1 , (aM2 , aM2 , aL2 )∗). It fulfillsthe conditions in Eq. (1): none of the players would benefitfrom changing only the own strategy. However, this wouldbe merely the case if P2 ‘threatens’ to provoke a collision byplaying aM2 as reaction to aM1 , and aL2 as reaction to aR

1 . Actu-ally carrying out this threat would yet not be the best responseof P2. P1 can assume this to be an unlikely behavior and thusa non-credible threat. Also experience proves that humansrarely collide. They avoid collisions rather than provokingthem. This example shows that in dynamic games the notionof a Nash equilibrium can be too weak [44]. For that reason,the stricter subgame-perfect equilibria is used for dynamicgames in this work. Thus, equilibria that imply the threatto provoke a collision are excluded. A Nash equilibrium issubgame-perfect if it constitutes a Nash equilibrium in everysubgame of the complete game. An example of a subgame isshown in Fig. 4. It consists of a node within the tree and all itssubsequent nodes. In our example, the only subgame-perfectNash equilibrium is s∗ = (aM∗

1 , (aM2 , aR2 , aM2 )∗).

From now on, we will slightly abuse the notation of strate-gies in dynamic games. Instead of giving a combination ofactions for each choice node, only the best response tra-jectory of the second player to the first player’s decision isstated. For example the subgame-perfect equilibrium reducesto s∗ = (aM∗

1 , aR∗2 ) = (sM∗

1 , sR∗2 ), where aR

2 it the bestresponse to aM1 (circled cost pair in Fig. 4).

In this context we want to discuss the link between gametheory and optimal control. As mentioned before, the prob-lem formulation of game theory results in a set of coupledoptimal control problems. Nevertheless, the specific com-bination of a dynamic game that contains only the binarycomponent J I−IV

i in Eq. (3) as interactive part can also bereformulated by several independent optimal control prob-lems. Therefore, all actions of the agent choosing first thatinevitably lead to a collision with a following agent areremoved. After that, it is sufficient that the agent choosingfirst only minimizes its independent cost component J I−IV,which is independent of the actions of the other agents. Theagents acting afterwards choose their trajectory based onindependent optimizations with the already played actionsas constraints.

4 Evaluation Method

We evaluate if the Nash equilibria in either of the proposedgame setups sufficiently reproduce the decision processduring human navigation. Or differently phrased: we areinterested inwhether or not Nash’s solutionmirrors a human-

123

340 Int J of Soc Robotics (2016) 8:331–351

Fig. 5 Experimental setup;colored papers marked possiblestart/goal positions, markerswere put on the subjects andtheir positions over time wasrecorded

Fig. 6 Experimental setup of anavigation scene. Both picturesshow scene 1C-A3 wherein thesubjects were repeatedly askedto walk over field ‘1’ to ‘C’, andover field ‘A’ to ‘3’, respectively.A subset of the recordedtrajectories is drawn into thepicture on the right

like navigation behavior. We consider a Nash equilibriumallocation to be human-like if it proposes the same—or atleast an alike—solution as a humanwould choose. The valid-ity of the Nash solution is assessed separately for both gamemodels, each in combination with one of the five cost func-tions.

This section presents the experimental setup used to cap-ture the motion data, the game setup, and the statisticalanalysis used to test the presented approach. The steps withtheir inputs and outputs are shown in Fig. 9. It illustrates thatthe purpose of the experiment is to capture human trajec-tories which can be used as action sets for the game setup.Importantly, the validation is based entirely on humanmotiondata to ensure that the action set contains valid human behav-ior. The next step—the game theoretic analysis—calculatesall allocations that constitute a Nash equilibrium. Then thevalidity of the Nash solution is tested by comparing theirsimilarity to human solutions by applying a Bootstrap test.

At the end of this section, an alternative decision model togame theory based on independent prediction is introduced.It serves as a further baseline for the performance evaluationof the proposed decision model.

4.1 Experimental Setup

In order to acquire a set of human walking trajectories, werepeatedly recorded the motion of two persons passing eachother. Thereby, the start and goal positions changed partly.As preparation three possible start and end positions weremarked with colored DinA4 papers on the floor and at eyelevel. The setup of the papers as well as the dimensions ofthe recording area is shown in Figs. 5 and 6. The distancebetween the edges of the papers was chosen to be 0.4 m.

During the experiment two participants facing each otherwere advised to walk over previously selected start and goalpositions. Thereby, they started one meter in front of theactual starting marker such that the acceleration phase wasbeyond the recorded area (compare Fig. 6). Start and goalpositions were known to both participants. Overall, 17 dif-ferent start/goal configurations were chosen such that thecovered distance in a whole was the same for each person.Thereby, the paths crossed in 10 out of 17 cases. All chosenconfigurations are listed in Fig. 10. Here, a configuration ofstart/goal positions including the two recorded trajectories iscalled scene and named according to the start and goal posi-tions. For example, the scene shown in Fig. 6 on the left isscene 1C-A3. In our previous work [80], we already showedthe existence of interaction in most of these scenes by com-paring them to free-space motions.

In order to create our database, eight healthy subjects(mean age ± SD: 27 ± 2.7 years) were recorded. Each ofthe 17 scenes was repeated ten times for two pairs of sub-jects, and five times for the other pairs leading to [in total:(4 · 10 · 17) + (4 · 5 · 17)] 1020 trajectories. The sequence ofscenes was randomized differently for all subjects.

The human motion was recorded with the vision basedmotion capture system Qualisys.3 Thereby, reflective mark-erswere put on each person and the positions of thesemarkerswere recorded over time (at 204 Hz). After that, the meanposition of the markers was calculated at each time stepfor each person and smoothed with a Butterworth filter (4thorder, 0.01 cutoff frequency [50]) in order to filter the torsooscillations caused by the steps. The distribution of the fivemarkers is shown in Fig. 5. It was chosen by following the

3 http://www.qualisys.com/.

123

Int J of Soc Robotics (2016) 8:331–351 341

Fig. 7 Illustration of theactions of game 1C-A3. Actionsthat are incorporated into actionset A are drawn solid, others aredashed. All trajectories goingfrom 1 to C out of the scenes1C-A3 and 1C-A2 constituteA1. The set A2 consists of alltrajectories going from A to 3out of the scenes 1C-A3 and2B-A3. Table 2 shows thecorresponding bi-matrix, thegrey area maps actions to entriesin the matrix

setup in [50]. However, we neglected the markers on theknee and feet such that the mean constitutes approximatelythe center of mass of the torso. The resulting discrete trajec-tories constitute the actions in the game theoretic setup.

4.2 Game Theoretic Setup

Each of the 17 scenes with different start/end configurationscan be represented as an individual game. They differ fromeach other by the action sets. In the following, the game setupis shown by the example of game 1C-A3 (Fig. 6). First, thestatic game is discussed. A bi-matrix is set up (Table 2) andthe components in Definition 1 are mapped:

1. The player set P consists of the two subjects.2. The action setA consists of the action setA1 andA2. In

the game 1C-A1, A1 contains all trajectories a j,1C1 that

P1 walked during the experiment starting at ‘1’ going to‘C’—not just the trajectories of all scenes 1C-A3. Fora better understanding compare with Fig. 7 and the cor-responding bi-matrix in Table 2. Here a subset of A1 isdrawn as solid lines in green (left, scenes 1C-A3) andblack (middle, scenes 1C-A2). This means, all the trajec-tories of other scenes with ’1C’, as 1C-A2 or 1C-B1, arealso part of the action set because they constitute validways to get from ‘1’ to ‘C’.Likewise, the action setA2 contains all trajectories a

j,A32

that were captured while P2 was walking from ‘A’ to‘3’. They are drawn as solid lines in green (left, scenes1C-A3) and black (right, scenes 2B-A3).

3. Each allocation s = (s j1 , s j2 ) = (a j,1C1 , a j,A3

2 ) is ratedwith a cost function as in Eq. (2) for each player. Thebi-matrix in Table 2 is constructed by using cost func-tion J IV. The collision radius R is chosen individuallyfor each pair of subjects by identifying the minimumrecorded distance from all simultaneous walks.

Table 2 Reduced bi-matrix of the game 1C-A3 (Fig. 7)

Reference trajectory allocations are marked bold, the Nash equilibriumis circled

Second, we model the game 1C-A3 dynamically (Defini-tion 2). The players, action sets and cost functions remain thesame as for the static game. In contrast, the strategy spacechanges because dynamic games can have a sequence ofactions. However, this sequence needs to be defined before-hand. According to the human avoidance studies in [55,61],the agent coming first adopts the trajectory less. This indi-cates that this agent chooses first while the other one reacts.Thus, we determine for each scene and each pair of subjectswhich subject was more often the one entering the recordedarea first. In Sect. 5, both options, that this player acts eitherfirst or second, are evaluated.

After setting up the game, the Nash equilibria are calcu-lated. In Table 2 the bi-matrix of the game 1C-A3 is shown,for clarity the action set is reduced. The only Nash equilib-riumof the (reduced) static game is circled: it is the allocations∗ = (a4∗,1C

1 , a2∗,A32 ). This corresponds to a pair of tra-

jectories, one trajectory for P1 and one trajectory for P2,respectively. It is denoted as equilibrium trajectory pair.

We assume that human interaction-aware decisionmakingduring navigation can be approximated with the Nash equi-

123

342 Int J of Soc Robotics (2016) 8:331–351

Fig. 8 Illustration of the recorded trajectories and sets used to analyzethe game 1C-B2 for one of the subject pairs

librium solutions from game theory. This is only true if theseequilibrium trajectory pairs constitute a provable human-likesolution. In order to test the assumption, real human solutionpairs are needed to serve as ground truth to which the equilib-rium pairs are compared. Such pairs will be called referencetrajectory pairs. They are the simultaneously recorded trajec-tories of a scene. For example, the subjectswere asked towalkfrom ’1’ to ’C’ and from ’A’ to ’3’, respectively. This leadsto a captured trajectory pair (a j,1C

1 , a j,A32 ). This trajectory

pair can be used as ground truth for game 1C-A3. In the fol-lowing, reference trajectory pairs are tagged with a tilde. Aseach scene was repeatedly captured during the experiment,several reference trajectory pairs exist for each game. Theset containing all reference trajectory pairs (a j,1C

1 , a j,A32 ) is

denoted asR1C-A3. The elements of its complementR1C-A3

are (a j,1C1 , a j,A3

2 ). Additionally, the set E1C-A3 is defined that

contains all equilibrium pairs (a j∗,1C1 , a j∗,A3

2 ) of a game. Its

elements can be elements of both R1C-A3 and R1C-A3. By

applying this to our example in Table 2, the sets areR1C-A3 ={(a1,1C1 , a1,A32 ), (a2,1C1 , a2,A32 ), (a3,1C1 , a3,A32 )} (bold pairs)

and E1C-A3 = {(a4∗,1C1 , a2∗,A3

2 )} (circled).For further illustration, the recorded trajectories (i.e.,

actions) of another game (1C-B2) are drawn in Fig. 8. Theactions are assigned different colors that represent the dif-

ferent trajectory pair sets. In this example one equilibriumtrajectory pair exists (drawn in violet). The reference trajec-tories are green and the remaining trajectories are black.

4.3 Similarity Measurement

In order to test the proposed approach, we check if the equi-librium trajectory pairs in E1C-A3 are more similar to the

reference pairs inR1C-A3 than the other pairs inR1C-A3. The

similarity is measured with the Dynamic Time Warping dis-tance δ. The distance δ is zero if two time series are identical,hence, the smaller the value of the distance the more simi-lar they are. The algorithm is used because it can comparetrajectories that differ in the number of time steps. Gillian etal. [25] explain how to apply it for multi-dimensional timeseries (e.g., trajectories).

The Dynamic TimeWarping distance between two trajec-tories is denoted as δ(ali , a

ji ), where ali is the tested reference

trajectory and a ji the one that is tested against. In order to

compare a reference trajectory pair to another pair of trajecto-ries, the twoDynamic TimeWarping distances are calculatedseparately and summed up. This leads to a trajectory pairdistance d:

d = δ(al,1C1 , a j,1C

1

)+ δ

(al,A32 , a j,A3

2

). (12)

Note that only trajectories of the same player are compared.All trajectory pair distances between the elements ofR1C-A3

and E1C-A3, R1C-A3 and R1C-A3are calculated respectively,

leading to two additional sets. They are the output of thesimilarity measurement step (compare Fig. 9):

– SetD1C-A3E ; contains all trajectory pair distances d of the

possible comparison between the elements of E1C-A3 andR1C-A3.

– SetD1C-A3R ; contains all trajectory pair distances d of the

possible comparison between the elements ofR1C-A3and

R1C-A3.

As mentioned above, only one pair of subjects has been con-sidered so far. Considering all pairs of subjects is done byrepeating the game setup and similaritymeasurement step foreach pair of subjects. The resulting distance sets are merged.For simplicity we denoted the merged sets as D1C-A3

E andD1C-A3

R , too.

4.4 Statistical Validation Method

After calculating the sets D1C-A3E and D1C-A3

R , we are inter-

ested in whether or not the values in D1C-A3E are mostly

smaller than the values inD1C-A3R . Therefore, the null hypoth-

esis H0 is defined to be:

123

Int J of Soc Robotics (2016) 8:331–351 343

Fig. 9 Pipeline of the evaluation showing the inputs and outputs of each step for the analysis of the game 1C-A3. In favor of clarity, the superscriptsof the sets that index the game (e.g., E1C-A3) are omitted

Null hypothesis H0: The median values of the distribu-tions from which the two samples DE and DR are obtainedare the same.

We test against this null hypothesis by computing the pvalue with a one-sided Bootstrap test [21] using a 5% signif-icance level. We use Bootstrap, a resampling method widelyused in statistical inference, because the true distributions areunknown and the considered sample sizes of some scenes aretoo small (in some cases <30) for inference based on the t-distribution.

Since 17 different scenes are regarded, 17 Bootstrap testsare necessary. In order to overcome themultiple testing prob-lem, the Benjamini-Hochberg procedure [8] was used toadjust the significance level of the p values.

4.5 Baseline Comparison: Prediction Based Decision

In order to further evaluate the game theoretic approach, itsperformance is compared to the performance of a predic-tion based decision model. Therefore, a model is used whereeach agent independently predicts the future motions of sur-rounding agents first and decides afterwards which trajectoryto take based on the prediction. It is a model that—like thepresented one—anticipates collisions and assumes humansto be more than merely reacting particles. However, it omitsthe reasoning about possible other motions of agents.

For the experimental setup of two persons passing eachother, the prediction based decision model is realized asfollows. Again the setup of the game 1C-A3 is used as exam-ple. Both persons know the current position, velocity andgoal of the other person. Based on this knowledge and theassumption that persons move with constant velocity to theirgoal, person P1 predicts the future trajectory of P2, and viceversa. Note, that this differs from a merely constant veloc-ity approach because the goal is known. Only the way tothe goal is predicted. The predictions are denoted as a1C,pred

1

and aA3,pred2 . Then P1 chooses a trajectory that minimizes

the cost function min(J1(aj,1C1 , aA3,pred2 )), ∀a j,1C

1 ∈ A1. P2

does likewise for min(J2(a1C,pred1 , a j,A3

2 )), ∀a j,A32 ∈ A2.

The output will be the trajectory pair (a j∗,1C1 , a j∗,A3

2 ). Thehuman-likeness of this decision is validated with the sameapproach as for the equilibrium trajectory pairs as illustratedin Fig. 9. The only difference is that, instead of using gametheory to decide on which trajectory pair to take, the predic-tion based decision models is used.

5 Results

The results of the statistical validation with the Bootstraptests will be presented in this section. It is followed by theresults of the alternative prediction based decisionmodel anda discussion and potential shortcomings.

5.1 Static Game Model

First, the results of the evaluation of the Nash equilibria instatic games are presented. Table 3 shows for how manytests (out of 17) the null hypothesis (see Sect. 4.4) wasrejected after using the Benjamini-Hochberg procedure. Allfive choices of the cost function Ji are listed. Additionally,the corresponding p values are shown in Table 6.

The best result was achieved using J Ii , the length of a tra-jectory. In this case, the equilibrium pairs are more similarto the reference pairs than other possible solutions for all 17tests.Allmedianvalues of the trajectory pair distances relatedto the equilibrium pairs are significantly smaller. For illus-tration, the confidence intervals of the medians are shown inFig. 10. They are also calculated with a Bootstrap test using a5 % significance level. Note that none of the confidence inter-vals overlap. This means, that the theory of Nash equilibriain static games using J Ii is indeed suitable to approximatethe decision process behind human avoidance maneuversbecause it chooses the same, or at least similar, trajectoriesas the subjects did. Almost as good performed the other pathbased cost function J IIi rating curvature; the null hypothesiswas rejected in 16 cases.

123

344 Int J of Soc Robotics (2016) 8:331–351

Table 3 Evaluation results for static game modeling

Input Cost function Ji H0 rejected(out of 17)

Path J Ii (Length) 17 (100%)

J IIi (Papadopoulos et al. [57]) 16 (94%)

Trajectory J IIIi (Time) 14 (82%)

J IVi (Mombaur et al. [50]) 13 (76%)

JVi (Pellegrini et al. [60]) 10 (59%)

Table 6 lists the p values of all 17 tests in detail (left column)

0 50 100 150 200

1: 2B-B2

2: 2B-A3

3: 3A-B2

4: 2B-A2

5: 3B-B2

6: 1C-A3

7: 3A-C1

8: 3A-B1

9: 1B-A3

10: 2A-B1

11: 1B-A2

12: 1A-A2

13: 3B-C3

14: 3B-C1

15: 1C-A2

16: 1C-B1

17: 2C-C1

game

trajectory pair distance d

DEDR

Fig. 10 Static game modeling. The 95% confidence intervals of themedians of the trajectory pair distances are shown for cost function J Ii(Length) (compare Table 3). DE referes to the similarity of the Nashequilibrium trajectory pairs to the ground truth, DR referes to the set ofremaining possible trajectory pairs, respectively. If they do not overlapthe difference is significant and H0 is rejected

Table 3 further reveals that the two cost functions regard-ing the path seem to represent the human behavior moreaccurately than the ones regarding trajectory cost. More-over, the elemental cost functions—i.e., cost for length J Iior time J IIIi —tend to achieve better results than the respec-tive learning based cost functions J IIi , J

IVi . Since they were

learned in free-space, these functions may be too specificfor the direct usage in an environment populated by humans.Both learned cost functions include either a cost for time orlength, however, also add a goal dependent cost. For exam-ple, J IVi charges an additional cost if the agent is not facingthe goal. One could argue that this goal driven behavior

becomes less important in such environments while mini-mizing time/length is still a prevalent aim of humans.

The worst result was achieved for JVi , which shares char-acteristics with the social force model. This cost functionoften leads to equilibrium pairs that are further apart than therespective pairs of the cost functions that merely considercollisions in the interactive component. Apparently, JVi setsto much emphasis on proximity when applied in a game the-oretic setup for the presented scenario.

5.2 Dynamic Game Model

If a scene is modeled as a dynamic game, we assume that thesubject whowasmore often the first one to enter the recordedarea is allowed to choose first. In the tables this player willbe marked with a circled one, hence, P 1©

i . In order to assessif our assumption holds and if the order makes a difference,each dynamic game is played twice; ones with the playerP 1©i choosing first and ones with P 1©

i choosing second. Afterthat, the human-likeness of the Nash solution is validated asdescribed previously. Table 4 summarizes the results and thecorresponding p values are listed in detail in Table 6.

The null hypothesis is rejected for the majority of the testsif P 1©

i chooses first. The sequence of which cost performsbest is similar to the static model, however, the number ofrejected null hypothesis is lower in each case. Only JVi per-forms similarly well in the static and dynamic setup, but isstill the weakest. The best result was again achievedwith costfunction J Ii (Length). Additionally, the results indicate thatgames with J Ii are the ones being the most affected by chang-ing the playing order. This result is in line with the studiesfrom Pettré et al. [61] which state that the person giving wayneeds to make a larger avoidance maneuver. Opposed to thatis cost function J IIIi ; here the number of rejected hypothesesslightly increases if P 1©

i chooses second.

5.3 Prediction Based Decision Model

The results for the validation of the decision model basedon constant velocity to the goal prediction is summarized inTables 5 and 6. Similar to the static game, the path basedcost functions perform better than the trajectory based ones,however, all cost functions perform clearly worse thanwithinthe static or dynamicmodel. Especially clear is the differencefor JVi : only for two scenes, out of 17, the suggested trajectorypairs are more similar to the reference pairs than a randomlypicked pair.

5.4 Discussion

The result that the prediction based decision model per-formed worse than the game theoretic decision model is inlinewith Trautman et al. [78]: it indicates that it is insufficient

123

Int J of Soc Robotics (2016) 8:331–351 345

Table 4 Evaluation results fordynamic game modeling

Input Cost function Ji H0 rejected (out of 17)

P1©

i 1st P1©

i 2nd

Path J Ii (Length) 13 (76%) 10 (59%)

J IIi (Papadopoulos et al. [57]) 12 (71%) 12 (71%)

Trajectory J IIIi (Time) 12 (71%) 13 (76%)

J IIIi (Mombaur et al. [50]) 12 (71%) 11 (65%)

JVi (Pellegrini et al. [60]) 10 (59%) 7 (41%)

Table 6 lists the p values of all 17 tests in detail (center and right column)

Table 5 Evaluation results for prediction based decision model

Input Cost function Ji H0 rejected(out of 17)

Path J Ii (Length) 10 (59%)

J IIi (Papadopoulos et al. [57]) 10 (59%)

Trajectory J IIIi (Time) 8 (47%)

J IVi (Mombaur et al. [50]) 8 (47%)

JVi (Pellegrini et al. [60]) 2 (12%)

tomerely include prediction, even if the goal of the surround-ing person is known. It is advantageous andmore human-liketo consider interaction-awareness. Game theory is a suitableway to formulate the reasoning about possible interaction ofactions and approximates the human decision process duringnavigation more accurately than the prediction based model.

The numbers in Tables 3 and 4 further imply that the staticgame is more accurate than the dynamic one. Nevertheless,there is no significant difference between any of the mediansof the twoDE sets in the static anddynamic case, respectively.A reason why the null hypothesis was less rejected in thedynamic case may be that the set E is smaller if only thesub-game perfect Nash equilibria are considered. This resultsin a smaller sample size and thus in a lower confidence bydetermining the median. This may be resolved by recordingmore subjects.

Notwithstanding the above, another reason why thedynamic model is less accurate may be the policy for eval-uating P 1©

i . It may be too restricted in cases where theplayer choosing first is not fixed but swaps repeatedly, thusis independent from the scene. This is omitted in the currentvalidation method. On the one hand, P 1©

i is merely the sub-ject who was more often the first within the recorded area,but not always. On the other hand, the equilibrium trajec-tory pairs in E are compared to all reference pairs in R of agame. This issue can be addressed by reducing the setR suchthat it only contains the trajectory pairs of a scene whereinthe player choosing first is indeed the one who entered therecorded area first. We yet refrained from doing so becausefor some tests the sample sizewould shrink toomuch tomakea reliable statement.

Surprisingly, the performance of the only cost functionthat includes proximity cost, i.e. social cost, is always theweakest. As mentioned above, this is probably because itfavors trajectories that are too far apart. However, social forcefeatures in cost functions are still worth to be considered. Asmentioned inSect. 2,Vasquez et al. [82] investigated differentcost features. They concluded that social forces showed thebest results for the learned scenes while being at the sametime the ones generalizing worst for unknown scenes. Thismay have also happened for the used data set.

Finally, a number of potential shortcomings need to beconsidered. First, the study was limited to two person games.Extending it to several persons should still maintain theresults because more persons means less collision-free tra-jectories. Only those are chosen as Nash equilibrium andare hence more similar to the ground truth. Second, the pre-sented models assume that humans only decide ones and,thus, a sequence of decisions or changing a decision is notcaptured in the model yet. An exact model should alwayschoose one the reference pairs. However, building an exactmodel would only work if every detail of decision makingduring navigation is known and included. A game may alsohave several equilibria and the theory of Nash is indifferenttowards the question which equilibrium the players shouldchoose eventually. Therefore, the accuracy of the model canbe further enhanced and potential extensions are discussedin the next section.

6 Extensions and Applications

The following section looks beyond the horizon of the pre-sented method. The first part of the section calls attention topossible extensions of the presented game theoretic models.The second part discusses if the results can be further appliedto robots since the evaluation focuses on humans.

6.1 Further Analysis and Potential Extensions

Apart from comparing static and dynamic games, we addi-tionally looked into another game theoretic solution conceptfor static games, the Pareto optimality [44]. A Pareto optimaloutcome is an allocation in which it is impossible to reduce

123

346 Int J of Soc Robotics (2016) 8:331–351

Table6

Tableliststhepvalues

ofeach

test

Gam

eStaticgamemod

elDyn

amicgamemod

elP

1 © i1s

tDynam

icgamemodel

P1 © i2n

d

JI i

JII i

JIII

iJIV i

JV i

JI i

JII i

JIII

iJIV i

JV i

JI i

JII i

JIII

iJIV i

JV i

1:2B

-B2

εε

εε

εε

εε

εε

.002

εε

εε

2:2B

-A3

εε

εε

.086

.001

.007

ε1

.032

.004

.999

ε1

.058

3:3A

-B2

εε

.010

.011

.165

ε.072

.012

.011

.782

.717

εε

ε.323

4:2B

-A2

εε

εε

.034

εε

.001

.001

.032

εε

εε

.092

5:3B

-B2

εε

εε

εε

εε

εε

εε

εε

ε

6:1C

-A3

.045

.006

.416

.267

.900

.317

.054

.730

.804

.993

.274

.962

.420

1.917

7:3A

-C1

.001

ε.002

.003

.001

.019

ε.004

.006

.001

.016

ε.002

.003

.835

8:3A

-B1

εε

.203

.211

.495

ε.090

.117

.114

.961

.042

.084

.402

.415

.497

9:1B

-A3

.002

.002

.031

.069

.006

ε.001

.066

.060

.007

.001

.002

0.32

.042

.006

10:2

A-B1

εε

εε

ε.124

εε

εε

.006

εε

εε

11:1

B-A

2.004

ε.001

ε.004

.004

.005

.238

.015

.004

.275

ε.016

.016

.033

12:1

A-A

2.001

ε.008

εε

.001

ε.006

.001

ε.001

ε.008

ε.007

13:3

B-C3

εε

εε

εε

εε

εε

εε

εε

ε

14:3

B-C1

.001

εε

ε.012

.258

.002

.007

ε.897

.657

.114

.070

ε.988

15:1

C-A

2.004

.177

.386

.002

.227

.006

.409

.396

.001

.980

.290

.488

.386

.897

.223

16:1

C-B1

.002

εε

.227

.195

.007

.068

.002

.281

.352

.022

εε

.232

.282

17:2

C-C1

.006

.021

.001

.022

.008

.065

.023

.026

.021

.007

.063

.023

.026

.021

.008

The

pvaluewas

calculated

with

aon

e-sidedBoo

tstrap

teston

a5%

sign

ificancelevel.Allpvalues

which

aregreaterthan

thesign

ificancelevelasfix

edwith

theBenjamini-Hochbergprocedure

aremarkedbo

ld(Tables3and4summarizethistable).V

alueswhich

aresm

allerthan

0.001aredepicted

bythesymbol ε

123

Int J of Soc Robotics (2016) 8:331–351 347

the cost of any player without raising the cost of at leastone other player. Thus, Pareto optimality has some notionof ‘fairness’, while the Nash concept assumes rational play-ers, which merely minimize their own cost. Nash equilibriaare mostly Pareto inefficient [34]. However, choosing a costfunction of the form as in Eq. (2) with Eq. (3) results in theset of Pareto optimal allocations being a subset of the Nashequilibrium allocations. For example, in the game in Table 1the cost of the Pareto optimal allocations are (3|1), (2|2) and(1|3). As a consequence, the statistical comparison betweenthe two sets of trajectory pair distances—one set using theNash concept, and one set using Pareto optimality—revealedno significant results for the four cost function using the inter-active component as defined in Eq. (3). Nevertheless, Paretooptimality is worth to be considered if further interactivecomponents are added to the cost function (e.g., keeping acomfortable distance). In this case the Nash equilibria andPareto optimal allocations coincide less and a comparisonis more expressive. Accordingly, additional tests reveal thatthe performance increases for JVi if Pareto efficiency is usedas solution concept: in 15 cases the null hypothesis can berejected (instead of 10 cases with Nash, see Table 3).

A consequence of the presentedmodel is that players needto coordinate their choices and “agree upon” an equilibrium.Rules on how humans come to an agreement are studiedin coordination games [18]. The agreement can be basedon facts like in cases where one of the equilibria is payoffsuperior or less risky, but also on social rules. An especiallyinteresting extension for navigation is to include traffic rulesinto the formulation.

The mentioned models and extensions all assume that theagents behave rationally in a sense that they minimize theirexpected cost. They also imply common knowledge (seeSect. 3.2). Researchers within different sub-fields of gametheory yet argue that the common knowledge assumption andthe conventionally defined rationality “is not characteristic ofhuman social interaction in general” [17]. An apparent casewhere common knowledge is hard to imply is if the numberof agents is too large. The conditions may also be violatedif it is required to plan far ahead in the future or if the prob-lem is complex (e.g., for games like Chess or Go). Theseare different facets of bounded rationality, which is deliber-ately omitted in this work. So far we focus on well-knownand studied game theoretic approaches. They are evaluatedfor the presented task and compared among each other and toprior work. Thinking at the application of navigating humansin populated environments, we assume the number of poten-tial players to be uncritical. Also the planning horizon whilewalking is within seconds rather than minutes. Nevertheless,game theoretic approaches that consider bounded rationalitymay further improve our model and are highly applicablefor crowd simulations. For an overview see [68]. Other rele-vant concerns are raised by behavioral/psychological game

theorists [2,13]. They refer to various experimental resultswherein people seemingly do not decide rational, mean-ing act differently as predicted by the theory of Nash. Therationality assumption is often questioned in social decisionproblems wherein prior beliefs or emotions influence thedecision. Both are hard to pin down in utility functions sincein the world we can usually only measure pecuniary payoffsor cost [17]. It is one of the greatest challenges to definea perfectly correct analytic model of the human decisionprocess. Camerer yet comments in [17] that “many weak-nesses of game theory are cured by new models that embodysimple cognitive principles, while maintaining the formal-ism and generality that make game theory useful”. We wantto introduce game theory as an effective method to modelinteractivity during navigation, doing so by starting with ele-mentary game models that are step-wise refined in futurework, among others with approaches from behavioral gametheory. A promising next step is for example the fairnessequilibrium [13] that namely includes human sense of fair-ness into the utilities.

6.2 Relevance for Robot Motion Planning amongHumans

This paper focuses on how humans navigate among humans,yet with the intent to apply gained insights to improve thenavigation of robots among humans. It needs to be inves-tigated if the results of this work can be directly appliedto the robot navigation problem in populated environmentssince humans may react differently towards a robot thantowards a human. Nevertheless, existing literature indicatesthat humans perceive and treat robots similar as humans tosome extent. Accordingly, humans already associate humanfeatures with inanimate devices. For example, they attributeemotions with robot motion patterns [69] and ascribe inten-tions to moving objects, even if they are merely geometricshapes [14,26,33]. Intriguingly, researches also exposed thata robot action is represented in a similar manner within thehuman brain as the respective human action, if only forhand gestures and humanoid grippers [84]. But even if arobot resembles a human only to some degree, human-likebehaviors or features are still advantageous. Thus, theMediaEquation Theory [64] and the associated research field ofsocial robotics show that the performance of human-robotcollaboration is enhanced when robots employ human-likebehaviors [12,20,35,45,77]. Thus, we are confident that ourconclusions can enhance the human-robot cooperation dur-ing navigation.

7 Conclusion

Theunderstanding and correctmodeling of interaction-awaredecision making of humans during navigation is crucial to

123

348 Int J of Soc Robotics (2016) 8:331–351

further evolve robotic systems that operate in human pop-ulated workspaces. This paper introduces non-cooperativegame theory and the Nash equilibrium as a framework tomodel the decision process behind human interaction-awarebehavior. A condition for the suitability of Nash’s theoryis that humans behave rationally in a sense that they aimto minimize their own cost. In this work, this assumptionwas implicitly validated for five different cost functions. Thegame theoretic approach was first proposed formally andwas then applied and validated for the problem of predict-ing the decision of multiple agents passing each other. Weshowed that the solution concept of Nash equilibria in gamespicks trajectories that are similar to the humans’ choice oftrajectories. Thereby, the best results were achieved with astatic game model in combination with a length based costfunction. Moreover, using elemental cost functions—basedsolely on the length of the trajectory or the time needed—tended to bemore accurate than the respective learning basedcost functions. The game theoretic approach was addition-ally compared with a prediction based decision model. Itanticipates collisions but omits the reasoning about possibleother motions of persons, i.e. the interaction-awareness. Theresults show that both presented game theoretic models out-perform the prediction based decision model. This furtherhighlights the need to include interaction-awareness into thedecision modeling.

The derived knowledge is helpful for a variety of roboticsystems, like future service robots that need to predict humanmotion more accurately or that need to move in a human-likeway. It is equally usable for the autonomous automobile nav-igation or for modeling the interaction during armmovementcoordination tasks.

Future work includes improving the results by using acost function that considers further interaction parameterslike social or traffic rules. Thus, other solution concepts(Pareto optimality, fairness equilibrium) can be validated andcompared to the Nash equilibrium. Additionally, one has toanalyze if humans converge mainly to a specific equilibrium.Coordination games would supply a promising frameworkfor this analysis. In order to consider uncertainties in thegame formulation (e.g., the cost function of the players ortheir goals) we plan to apply Bayesian games and run exper-iments in which the players do have imperfect informationabout the intentions of the other players.Moreover, we intendto implement a game theory basedmotion planner and to con-duct human-human/human-robot experiments. They furtherevaluate the future approach and the question on howhumansperceive robots during navigation.

Acknowledgments This work was supported in part by theTechnische Universität München - Institute for Advanced Study(www.tum-ias.de), funded by the German Excellence Initiative and inpart within the ERC Advanced Grant SHRINE Agreement No. 267877(www.shrine-project.eu).

OpenAccess This article is distributed under the terms of theCreativeCommons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution,and reproduction in any medium, provided you give appropriate creditto the original author(s) and the source, provide a link to the CreativeCommons license, and indicate if changes were made.

References

1. Alonso-Mora J, Breitenmoser A, Rufli M, Beardsley P, Sieg-wart R (2013) Optimal reciprocal collision avoidance for multiplenon-holonomic robots. In: Martinoli A, Mondada F, Correll N,Mermoud G, Egerstedt M, Hsieh MA, Parker LE, Sty K (eds) Dis-tributed autonomous robotic systems. Springer, Berlin Heidelberg,pp 203–216

2. Attanasi G, Nagel R (2008) A survey of psychological games:theoretical findings and experimental evidence. In: Innocenti A,Sbriglia P (eds) Games, rationality and behavior. Essays on behav-ioral game theory and experiments. Palgrave Macmillan, NewYork, pp 204–232

3. Basar T, Olsder GJ (1999) Dynamic noncooperative game theory,2nd edn. SIAM, Philadelphia

4. Basar T, Bernhard P (1995) H-infinity optimal control and relatedminimax design problems. Birkhäuser, Basel

5. Basili P, Saglam M, Kruse T, Huber M, Kirsch A, Glasauer S(2013) Strategies of locomotor collision avoidance. Gait Posture37(3):385–390

6. van Basten B, Jansen S, Karamouzas I (2009) Exploiting motioncapture to enhance avoidance behaviour in games. In: Egges A,Geraerts R, Overmars M (eds) Motion in Games. Springer, Berlin,pp 29–40

7. Bauer A, Klasing K, Lidoris G, Mühlbauer Q, Rohrmüller F, Sos-nowski S, Xu T, Kühnlenz K, Wollherr D, Buss M (2009) Theautonomous city explorer: towards natural human-robot interac-tion in urban environments. Int J of Soc Robot 1(2):127–140

8. Benjamini Y, Hochberg Y (1995) Controlling the false discoveryrate: A practical and powerful approach to multiple testing. J R StatSoc Series B Stat Methodol 57:289–300

9. van den Berg J, Patil S, Sewall J, Manocha D, Lin M (2008) Inter-active navigation of multiple agents in crowded environments. In:Proceedings of the ACM SIGGRAPH symposium on interact 3Dgraph games, pp 139–147

10. van den Berg J, Guy S, Lin M, Manocha D (2011) Reciprocal n-body collision avoidance. In: Pradalier C, Siegwart R, Hirzinger G(eds) Robotics research. Springer, Berlin, pp 3–19

11. Bitgood S, Dukes S (2006) Not another step! Economy of move-ment and pedestrian choice point behavior in shopping malls.Environ Behav 38(3):394–405

12. Breazeal C, Kidd C, Thomaz A, Hoffman G, Berlin M (2005)Effects of nonverbal communication on efficiency and robustness inhuman-robot teamwork. In: Proceedings of IEEE/RSJ internationalconference on intelligent robots and systems, pp 708–713

13. CamererC (2003)Behavioral game theory: experiments in strategicinteraction. Princeton University Press, Princeton

14. Castelli F, Happé F, Frith U, Frith C (2000) Movement and mind: afunctional imaging study of perception and interpretation of com-plex intentional movement patterns. Neuroimage 12(3):314–325

15. CinelliM, Patla A (2007) Travel path conditions dictate themannerin which individuals avoid collisions. Gait Posture 26(2):186–193

16. Cinelli M, Patla A (2008) Locomotor avoidance behaviours duringavisually guided task involving an approachingobject.Gait Posture28(4):596–601

123

Int J of Soc Robotics (2016) 8:331–351 349

17. Colman A (2003) Cooperation, psychological game theory, andlimitations of rationality in social interaction. Behav Brain Sci26(02):139–153

18. Cooper R (1999) Coordination games. Cambridge UniversityPress, Cambridge

19. Csibra G, Gergely G, Bıró S, Koos O, Brockbank M (1999) Goalattribution without agency cues: the perception of pure reasonininfancy. Cognition 72(3):237–267

20. Dragan AD, Bauman S, Forlizzi J, Srinivasa SS (2015) Effectsof robot motion on human-robot collaboration. In: Proceedings ofACM/IEEE international conference on human-robot interaction,pp 51–58

21. Efron B, Tibshirani R (1993) An introduction to the bootstrap.Chapman & Hall/CDC, Washington D.C

22. Fiorini P, Shillert Z (1998) Motion planning in dynamic environ-ments using velocity obstacles. Int J Robot Res 17:760–772

23. Ganebny S, Kumkov S, Patsko V (2006) Constructing robust con-trol in game problems with linear dynamics. In: Petrosjan L,Mazalov V (eds) Game theory and applications, 11th edn. NovaScience Publishers, New York

24. Ghazikhani A, Mashadi HR, Monsefi R (2010) A novel algorithmfor coalition formation in multi-agent systems using cooperativegame theory. In: Procedings of IEEE Iranian conference on elec-trical engineering, pp 512–516

25. Gillian N, Knapp B, OModhrain S (2011) Recognition of mul-tivariate temporal musical gestures using n-dimensional dynamictime warping. In: Proceedings of the international conference onnew interfaces for musical expression, pp 337–342

26. Heider F, Simmel M (1944) An experimental study of apparentbehavior. Am J Psychol 57:243–259

27. Heïgeas L, Luciani A, Thollot J, Castagné N (2003) A physically-based particlemodel of emergent crowd behaviors. In: Proceedingsof GraphiCon

28. Helbing D, Molnár P (1995) Social force model for pedestriandynamics. Phys Rev E 51:4282–4286

29. Henry P, Vollmer C, Ferris B, Fox D (2010) Learning to navigatethrough crowded environments. In: Proceedings of the IEEE inter-national conference on robotics and automation, pp 981–986

30. Hoogendoorn S, Bovy P (2003) Simulation of pedestrian flowsby optimal control and differential games. Optim Control ApplMethods 24(3):153–172

31. Huber M, Su YH, Krüger M, Faschian K, Glasauer S, HermsdörferJ (2014) Adjustments of speed and path when avoiding collisionswith another pedestrian. PloS One 9(2):e89,589

32. Huntsberger T, Sengupta A (2006) Game theory basis for control oflong-lived lunar/planetary surface robots. Auton Robots 20(2):85–95

33. Johnson S (2003) Detecting agents. Philos Trans R Soc B358(1431):549–559

34. Kameda H, Altman E, Touati C, Legrand A (2012) Nash equilib-rium based fairness. Math Methods Oper Res 76(1):43–65

35. Kato Y, Kanda T, Ishiguro H (2015) May i help you?: Designof human-like polite approaching behavior. In: Proceedings ofACM/IEEE international conference on human-robot interaction,pp 35–42

36. Kim B, Pineau J (2013) Human-like navigation: Socially adap-tive path planning in dynamic environments. In: RSS workshop oninverse optimal control & robot learning from demonstrations

37. KlugeB, Prassler E (2004)Reflective navigation: individual behav-iors and group behaviors. In: Proceedings of IEEE internationalconference on robotics and automation, pp 4172–4177

38. Kretzschmar H, Kuderer M, Burgard W (2014) Learning to pre-dict trajectories of cooperatively navigating agents. In: Proeedingsof IEEE international conference on robotics and automation, pp4015–4020

39. Kruse T, Pandey A, Alami R, Kirsch A (2013) Human-aware robotnavigation: a survey. Robot Auton Syst 61(12):1726–1743

40. Kuderer M, Kretzschmar H, Burgard W (2013) Teaching mobilerobots to cooperatively navigate in populated environments. In:Proceedings of IEEE/RSJ international conference on intelligentrobots and systems, pp 3138–3143

41. LaValle S (2006) Planning algorithms. Cambridge UniversityPress, Cambridge

42. LaValle S,HutchinsonS (1993)Game theory as a unifying structurefor a variety of robot tasks. In: Proceedings of IEEE internationalsymposium on intelligent control, pp 429–434

43. LernerA,ChrysanthouY, LischinskiD (2007)Crowds by example.Comput Graph Forum 26(3):655–664

44. Leyton-Brown K, Shoham Y (2009) Essentials of game theory: aconcise multidisciplinary introduction. Morgan & Claypool, SanRafael

45. Lichtenthäler C, Lorenzy T, Kirsch A (2012) Influence of legibilityon perceived safety in a virtual human-robot path crossing task. In:Proceedings of IEEE international symposium on robot and humaninteractive communication, pp 676–681

46. Luber M, Spinello L, Silva J, Arras K (2012) Socially-aware robotnavigation: A learning approach. In: Proceedings of IEEE/RSJinternational conference on intelligent robots and systems, pp 902–907

47. McNeill A (2002) Energetics and optimization of human walkingand running: the 2000 raymond pearl memorial lecture. Am J HumBiol 14(5):641–648

48. Meng Y (2008) Multi-robot searching using game-theory basedapproach. Int J Adv Robot Syst 5(4):341–350

49. Mitchell I, Bayen A, Tomlin C (2005) A time-dependent hamilton-jacobi formulation of reachable sets for continuous dynamic games.IEEE Trans Autom Contr 50(7):947–957

50. Mombaur K, Truong A, Laumond JP (2009) From human tohumanoid locomotion an inverse optimal control approach. AutonRobots 23(3):369–383

51. Myerson R (1991) Game theory: analysis of conflict. Harvard Uni-versity Press, Cambridge

52. Nash J (1950) Non-cooperative games. PhD thesis, Princeton Uni-versity

53. de Nijs R, Julia M, Mitsou N, Gonsior B, Kühnlenz K,Wollherr D,Buss M (2011) Following route graphs in urban environments. In:Proceedings of IEEE international symposium on robot and humaninteractive communication, pp 363–368

54. Olivier AH,Marin A, Crétual A, Pettré J (2012)Minimal predicteddistance: a common metric for collision avoidance during pairwiseinteractions between walkers. Gait Posture 36(3):399–404

55. Olivier AH, Marin A, Crétual A, Berthoz A, Pettré J (2013) Col-lision avoidance between two walkers: role-dependent strategies.Gait Posture 38(4):751–756

56. Ondrej J, Pettré J, Olivier AH, Donikian S (2010) A synthetic-vision based steering approach for crowd simulation. ACM TransGraph 29(4):123:1–123:9

57. Papadopoulos A, Bascetta L, Ferretti G (2013) Generation ofhuman walking paths. In: Proceedings of IEEE/RSJ internationalconference on intelligent robots and systems, pp 1676–1681

58. Papavassilopoulos G, Safonov M (1989) Robust control designvia game theoretic methods. In: IEEE conference on decision andcontrol, pp 382–387

59. Pelechano N, Allbeck J, Badler N (2007) Controlling individualagents in high-density crowd simulation. In: Proceedings of ACMSIGGRAPH/Eurographics symposium on computer animation, pp99–108

60. Pellegrini S, Ess A, Schindler K, Van Gool L (2009) You’ll neverwalk alone: modeling social behavior for multi-target tracking. In:Proceedings of IEEE international conference on computer vision,pp 261–268

123

350 Int J of Soc Robotics (2016) 8:331–351

61. Pettré J, Ondrej J, Olivier AH, Cretual A, Donikian S (2009)Experiment-based modeling, simulation and validation of inter-actions between virtual walkers. In: Proceedings of ACM SIG-GRAPH/Eurographics symposium on computer animation, pp189–198

62. Philippsen R, Siegwart R (2003) Smooth and efficient obstacleavoidance for a tour guide robot. In: Proceedings of IEEE interna-tional conference on robotics and automation, pp 446–451

63. Prassler E, Scholz J, Strobel M (1998) MAid: mobility assistancefor elderly and disabled people. In: Proceedings on conference ofindustrial electronics society

64. Reeves B, Nass C (1996) How people treat computers, televi-sion, and new media like real people and places. CSLI Publica-tions/Cambridge University Press, Stanford/New York

65. Reynolds C (1999) Steering behaviors for autonomous characters.In: Game developers conference, pp 763–782

66. Rios-Martinez J, Spalanzani A, Laugier C (2014) From proxemicstheory to socially-aware navigation: a survey. Int J of Soc Robot7(2):137–153

67. Roozbehani H, Rudaz S, Gillet D (2009) A hamilton-jacobiformulation for cooperative control ofmulti-agent systems. In: Pro-ceedings of IEEE international conference on systems, man andcybernetics, pp 4813–4818

68. Rubinstein A (1998) Modeling bounded rationality. MIT Press,Cambridge

69. Saerbeck M, Bartneck C (2010) Perception of affect elicited byrobot motion. In: Proceedings of ACM/IEEE international confer-ence on human-robot interaction, pp 53–60

70. Seder M, Petrovic I (2007) Dynamic window based approach tomobile robot motion control in the presence of moving obstacles.In: Proceedings of IEEE international conference on robotics andautomation, pp 1986–1991

71. Shiomi M, Zanlungo F, Hayashi K, Kanda T (2014) Towards asocially acceptable collision avoidance for a mobile robot navigat-ing among pedestrians using a pedestrian model. Int J Soc Robot6(3):443–455

72. Sisbot EA,Marin-Urias LF, Broqure X, Sidobre D, Alami R (2010)Synthesizing robot motions adapted to human presence—a plan-ning and control framework for safe and socially acceptable robotmotions. Int J Soc Robot 2(3):329–343

73. Skrzypczyk K (2005) Game theory based task planning in multirobot systems. Int J Simulat 6(6):50–60

74. Snape J, van den Berg J, Guy S, Manocha D (2010) Smooth andcollision-free navigation for multiple robots under differential-drive constraints. In: Proceedings of IEEE/RSJ international con-ference on intelligent robots and systems, pp 4584–4589

75. SparrowW,Newell K (1998)Metabolic energy expenditure and theregulation ofmovement economy. Psychon Bull Rev 5(2):173–196

76. Sud A, Gayle R, Andersen E, Guy S, Lin M, Manocha D(2007) Real-time navigation of independent agents using adaptiveroadmaps. In: Proceedings of ACM symposium on virtual realitysoftware and technology, pp 177–187

77. Takayama L, Dooley D, Ju W (2011) Expressing thought: improv-ing robot readability with animation principles. In: Proceedingsof ACM international conference on human-robot interaction, pp69–76

78. Trautman P, Ma J, Murray R, Krause A (2015) Robot navigation indense human crowds: statistical models and experimental studiesof human-robot cooperation. Int J Robot Res 34(3):335–356

79. Treuille A, Cooper S, Popovic Z (2006) Continuum crowds. ACMTrans Graph 25(3):1160–1168

80. Turnwald A, Olszowy W, Wollherr D, Buss M (2014) Interac-tive navigation of humans from a game theoretic perspective. In:Proceedings of IEEE/RSJ international conference on intelligentrobots and systems, pp 703–708

81. Urmson C, Baker C, Dolan JM, Rybski P, Salesky B, WhittakerWL, Ferguson D, Darms M (2009) Autonomous driving in traffic:boss and the urban challenge. AI Mag 30:17–29

82. Vasquez D, Okal B, Arras K (2014) Inverse reinforcement learningalgorithms and features for robot navigation in crowds: an exper-imental comparison. In: Proceedings of IEEE/RSJ internationalconference on intelligent robots and systems, pp 1341–1346

83. Vidal R, Shakernia O, Kim J, ShimD, Sastry S (2002) Probabilisticpursuit-evasion games: theory, implementation, and experimentalevaluation. IEEE Trans Robot Autom 18(5):662–669

84. Wykowska A, Chellali R, Al-AminMM,Müller H (2014) Implica-tions of robot actions for human perception. How do we representactions of the observed robots? Int J Soc Robot 6(3):357–366

85. Zanlungo F, Ikeda T, Kanda T (2011) Social force model withexplicit collision prediction. EPL 93(6):68,005:1–68,005:6

86. Zhang H, Kumar V, Ostrowski J (1998) Motion planning withuncertainty. In: Proceedings of IEEE international conference onrobotics and automation, pp 638–643

87. Zhu M, Otte M, Chaudhari P, Frazzoli E (2014) Game theoreticcontroller synthesis for multi-robot motion planning part I: tra-jectory based algorithms. In: Proceedings of IEEE internationalconference on robotics and automation, pp 1646–1651

88. Ziebart B (2010) Modeling purposeful adaptive behavior with theprinciple ofmaximum causal entropy. PhD thesis, CarnegieMellonUniversity

Annemarie Turnwald received the Diplom-Ingenieur degree fromTechnical University Munich, Germany, in 2011. She is currentlyworking towards the Doctoral degree with the Chair of AutomaticControl Engineering, Department of Electrical and Computer Engi-neering, Technical UniversityMunich, Germany, working in the field ofautonomous robot navigation in urban environment. Her research inter-ests include social robot navigation, interaction-aware decisionmaking,and game theory.

Daniel Althoff received the Diplom-Ingenieur degree and the Doctorof Engineering degree from Technical University Munich, Germany, in2008 and 2013, respectively. He worked on motion planning and safetyassessment for autonomous systems and its application tomobile robotsand highly automated vehicles. From 2013 to 2015 he joined the FieldRobotics Center at theCarnegieMellonUniversity, Pittsburgh, Pennsyl-vania, as a postdoctoral fellow. There, his work focused on developinga safe navigation framework for full-scale helicopters.

DirkWollherr received theDiplom-Ingenieur degree and theDoctor ofEngineering degree, both in electrical engineering, and the Habilitationdegree from Technical University Munich, Germany, in 2000, 2005,and 2013, respectively. From 2001 to 2004, he was a Research Assis-tant with the Control Systems Group, Technische Universität Berlin,Germany. In 2004, he was with the Yoshihiko Nakamura Laboratory,The University of Tokyo, Japan. Since 2014, he has been a Profes-sor with the Chair of Automatic Control Engineering, Department ofElectrical and Computer Engineering, Technical University Munich.His research interests include automatic control, robotics, autonomousmobile robots, humanrobot interaction, and humanoid walking.

Martin Buss received the Diplom-Ingenieur degree in Electrical Engi-neering in 1990 from the Technical University Darmstadt, Germany,and the Doctor of Engineering degree in Electrical Engineering fromthe University of Tokyo, Japan, in 1994. In 2000 he finished his habil-itation in the Department of Electrical Engineering and Information

123

Int J of Soc Robotics (2016) 8:331–351 351

Technology, Technical University Munich, Germany. As a postdoctoralresearcher he stayed with the Department of Systems Engineering,Australian National University, Canberra, Australia, in 1994/5. From1995-2000 he has been senior research assistant at the Chair of Auto-matic Control Engineering, Department of Electrical Engineering andInformation Technology, Technical University Munich, Germany. Hehas been appointed full professor, head of the control systems group, anddeputy director of the Institute of Energy and Automation Technology,

Faculty IV—Electrical Engineering and Computer Science, Techni-cal University Berlin, Germany, from 2000 to 2003. Since 2003 heis full professor (chair) at the Chair of Automatic Control Engineering,Technical University Munich, Germany. Martin Buss research inter-ests include automatic control, haptics, optimization, nonlinear, hybriddiscrete-continuous systems, and robotics.

123