Walking in Unknown Environments —

a Step Towards More Autonomy

Thomas Buschmann∗, Sebastian Lohmeier∗, Markus Schwienbacher∗, Valerio Favot∗ and Heinz Ulbrich∗

Felix von Hundelshausen∗∗, Gerhard Rohe∗∗ and Hans-Joachim Wuensche∗∗

∗Institute of Applied Mechanics, Technische Universitat Munchen, 85748 Garching, Germany

{buschmann,lohmeier,schwienbacher,favot,ulbrich}@amm.mw.tum.de∗∗Institute for Autonomous Systems Technology, University of the Bundeswehr Munich, 85577 Neubiberg, Germany

{felix,gerhard.rohe,joe.wuensche}@unibw.de

Abstract—We present a new approach to autonomous nav-igation for humanoid robots in previously unknown environ-ments. Our method uses only on-board cameras and doesnot rely on color coding, textured surfaces or predefinedobject models. We propose using a reactive approach towardsnavigation that does not require building an explicit modelof the environment. The basic idea of the navigation systemis to check a set of precalculated feasible trajectories (calledtentacles) for passability instead of searching for a path withina map. The computer vision system was combined with ourreal-time trajectory planning and control, enabling our robotLOLA to safely navigate in previously unknown environments.

I. INTRODUCTION

The development of humanoid robots is motivated by the

hope of building machines capable of performing useful tasks

anywhere a human could. This requires both a robot capable

of reliably executing a wide range of walking motions and

a computer vision and path planning system that can choose

safe paths in complex, previously unknown environments.

Despite significant advances, the basic technology of fast

and reliable biped locomotion is still an active research area.

We have developed the robot LOLA (Fig. 1) in order to

address specific issues of fast and autonomous biped walking.

The walking control and the robot itself were developed at

the Institute of Applied Mechanics and the computer vision

system at the Institute of Autonomous Systems Technology.

Weighing approximately 60 kg, the robot is 180 cm tall

and has 25 actuated degrees of freedom (DoFs) in a re-

dundant configuration, including 7-DoF legs with toe joints.

The mechanical structure is characterized by an extremely

lightweight design with high effective stiffness. The actua-

tors combine highly dynamic brushless servo motors with

precision gearings and sensors into a compact package.

Moreover, the resultant inertia of the legs is minimized by a

sophisticated design of the structure and drive mechanisms,

resulting in superior acceleration behavior. The sensor layout

comprises angular sensors for direct measurement of the

joint angles, a high-precision inertial measurement system in

the upper body and force/torque sensors in the feet. Further

details on the mechatronic system are given in Section III.

The control system running on the on-board PC enables

walking in arbitrary directions and along curved paths. The

reference path along which the robot should walk and where

the robot should look is determined by the computer vision

system. These high-level commands are converted into an

equivalent sequence of walking steps and a viewing target.

To obtain stable reference trajectories, a boundary value

problem for a simplified robot model is solved in real-time.

A stabilizing controller modifies the reference trajectories

based on data from the force/torque sensors and the inertial

measurement system. In Section IV the walking control is

presented in more detail, with a focus on the components

necessary for autonomous locomotion.

Fig. 1. Photograph of the 25-DoF humanoid walking robot LOLA

Most previous work on autonomous locomotion has fo-

cused on building environment models in which a planner

can search for feasible paths that are then sent to the robot

controller [1]–[4]. This requires a complex and computation-

ally expensive technique for simultaneous localization and

mapping (SLAM). We propose a more reactive approach.

Using on-board cameras, precalculated local trajectories are

classified as passable or obstructed. Interestingly, a stereo vi-

2010 IEEE-RAS International Conference on Humanoid RobotsNashville, TN, USA, December 6-8, 2010

978-1-4244-8690-8/10/$26.00 ©2010 IEEE 237

sion algorithm emerged that is quite different to conventional

stereo vision methods. We do not calculate dense depth maps,

we do not even calculate disparities. We just check whether

the image area corresponding to a potential path, the “visual

tentacle”, fulfills certain expectations. A detailed explanation

of this method is given in Section V.

Using this system, LOLA safely navigated among previ-

ously unknown obstacles at the Hannover Messe 2010 during

more than 25 presentations.

II. RELATED WORK

In this section we discuss some related research on au-

tonomously walking humanoid robots with a focus on large

robots using only on-board sensing.

The humanoid robot H7 developed by INOUE et al. [5]

could find potential walking directions by first calculating

a depth-map [1] in real-time using stereo-vision [6]. Areas

protruding from the floor plane, which are estimated using a

Hough-transform plane segment finder [3], are considered to

be blocked. H7 could also autonomously follow an orange

ball using a color-blob tracker [7]. However, this approach is

limited to environments with sufficient structural information

and/or color-coded objects.

In 2003 PFEIFFER and SCHMIDT presented experiments

in autonomous walking based on on-board sensing using

the robot JOHNNIE [8], [9]. Visual feedback was provided

by a combination of line-based scene analysis and real-

time feature tracking. In combination with a reactive step

sequence planner the robot could step over or walk around

obstacles. However, since all objects are assumed to have

edges and these edges are assumed to be either vertical or

horizontal, obstacles in the environment cannot have arbitrary

shapes.

Experiments on autonomous walking using on-board vi-

sion were also reported for Sony’s QRIO [2]. Similar to H7’s

system, the robot converts stereo data into 3-D range data.

The ground plane is determined using a slightly different

form of the Hough based method proposed in [3]. Navigation

was then performed using an A* based path planning algo-

rithm on an occupancy grid centered on the robot [2]. Using

an improved plane finder and stair detection algorithm [10],

QRIO was capable of autonomously navigating in an indoor

environment with no structural assumptions about obstacle

geometry [4]. However, in these experiments the environment

also provided sufficient image structure for dense stereo.

Experiments in autonomous walking using the HRP-2

robot were reported by MICHEL [11]. The robot is able to

climb stairs and avoid obstacles using on-board vision in

combination with a step sequence and swing leg trajectory

planner. While planner and controller have no knowledge of

the object geometry, the tracker uses a model for estimating

the object’s 6-D pose.

Recently, CHESTNUTT reported results on autonomous

and semi-autonomous navigation of a humanoid robot [12].

On the planning level, the approach is very similar to the

work mentioned above. The environment is represented by

a filtered height map calculated from the 3-D point cloud

provided by an on-board laser range scanner. The filtering

process identifies planes that are used to find viable footstep

locations. The robot can be guided by specifying either a

goal position, a 2-D guide path or by using a joystick.

Since a laser-range scanner is used, no particular texture,

shape or color of the objects is required. Moreover, the

system requires no particular object shapes and uses no prior

knowledge about the environment.

In general, robot navigation can be divided into global

and local methods. Global approaches, which are popular in

humanoid robotics, rely on building global maps in which a

planner can search for a feasible path. In contrast, local or

reactive approaches use only local data to generate control

commands for the robot [13], [14]. The key advantage of

local methods over global ones is their low computational

complexity, which is very important in time critical sce-

narios, e.g., when moving in a dynamic environment. The

drawback is that local methods usually do not find the

globally optimal path and can get trapped in local minima.

A local method related to our approach is the “dynamic

window approach” proposed by FOX for the wheeled robot

RHINO [14]. Steering commands are generated for a short

time interval by searching within a dynamic subset consisting

of velocities reachable within a short time interval and

admissible by the robot’s dynamics. Obstacle avoidance is

achieved by removing velocities that would lead to collisions

with objects detected by the robot’s sonar sensors.

VON HUNDELSHAUSEN evaluates sensor data on the basis

of a pre-calculated set of trajectories (called “tentacles”) the

robot could theoretically execute. A reference trajectory is

then chosen by checking which tentacles are passable, rather

than first building a map of the environment and planning

paths in that map. This so-called “tentacles”-approach was

originally designed for a multi-beam LIDAR (Velodyne)

sensor mounted on the vehicle platform MuCAR-3 [13].

Because of the promising results with MuCAR-3, the same

approach was adopted for LOLA’s computer vision system.

III. MECHATRONIC SYSTEM

LOLA’s fundamental design concept relies on the “clas-

sical” approach to building biped robots, using well-proven

and state-of-the-art mechatronics technology. Yet, inspiration

from biological sciences is applied wherever appropriate. Fo-

cusing on vision-guided autonomous locomotion, this section

can only emphasize the key aspects of LOLA’s mechatronic

system. The reader is referred to [15] for a detailed descrip-

tion of the robot hardware and the underlying design concept.

After emphasizing the key aspects to achieve high dynamic

performance and agility of the locomotor system (Section III-

A), the design of the stereo robotic head is presented in

Section III-B and the computer architecture is introduced in

Section III-C.

A. Basic Design Objectives for High Dynamic Performance

Higher walking speeds lead to large step lengths at high

step frequencies. Moreover, if the robot moves autonomously

in realistic scenarios, sudden and arbitrary changes of the

238

walking speed and direction may occur. For both tasks the

robot benefits from a greater acceleration capability of the

locomotor system. Thus, the judicious design of the legs and

their drive mechanisms is important. In summary, the basic

design objectives are:

• an extremely lightweight design, considering sufficient

static and dynamic stiffness of the mechanical structure

and drive mechanisms,

• actuators with high power density,

• a mass distribution in the legs optimized for low-inertia

and maximal height of the robot’s center of mass (CoM)

and

• a sophisticated foot dynamics concept for effective

impact attenuation.

The extremely lightweight design of the mechanical struc-

ture with high effective stiffness and natural frequencies

contributes to better dynamic properties, which are usually

penalized by added weight. Therefore, the major links are de-

signed as aluminum investment castings with a Monocoque-

like structure. Numerical modal analysis is adopted to assess

the dynamic properties of the mechanical structure. More-

over, the elastodynamic behavior of the drive mechanisms

is analyzed to avoid resonance oscillations and achieve

sufficient bandwidths and fast response times of the speed

control loops.

The joint actuators with high power density [16] form

a basis for high dynamic performance and acceleration

behavior of the robot. In order to actually benefit from

their performance, special attention is paid to the mass

distribution of the locomotor system. The drive mechanisms

of the knee and ankle joints place the actuators in the hip-

thigh area to minimize the inertia forces associated with

acceleration/deceleration of the leg links. Driving the ankle

joint in a parallel arrangement results in a more uniform

load distribution among the actuators and reduces the average

motor loads significantly. As a result, superior acceleration

behavior and reduced effects of varying inertia are achieved,

compared to other leg architectures, where the actuators are

typically arranged coaxially with the articulated joints.

In addition, the CoM height is increased, which reduces

the angular momentum about the fore-aft axis and, ulti-

mately, enlarges the stability margins available for balance

control.

The dynamic performance is further affected by the con-

straints associated with unilateral foot-ground contact. The

foot dynamics concept therefore ensures reliable contact on

different surfaces. At the same time, the feet incorporate

viscoelastic contact layers and passive heel segments with

hydraulic shock absorbers for effective impact attenuation at

initial contact [15].

B. Stereo Robotic Head

Most humanoid robots have rather simple head designs

where a stereo camera pair is mounted on a 2-DoF pan-tilt

unit, for example the HRP robots [17], [18] and JOHNNIE

[9]. JOHNNIE’s head was operated as a self-contained unit

within the vision and path planning system, hence the gaze

1

1

2

3

4

5

6

7

8

9

1 Camera 6 Tilt actuator2 Tilt segment 7 Parallel crank mechanism3 Camera receptacle 8 Vergence actuator4 Pan motor 9 Double slider-crank linkage5 Pan gearbox

Fig. 2. Mechanical design of the 3-DoF camera head

direction is controlled independently from the robot posture.

Few humanoids show more complex designs, for example

ARMAR-III [19].

Fig. 2 presents the mechanical design of LOLA’s stereo

camera head. The head mechanics comprise a 2-DoF pan-

tilt unit whose torque/speed capabilities allow smooth pursuit

movements, as well as the generation of human-like saccadic

camera motions. In addition, a 1-DoF mechanism is imple-

mented for camera vergence adjustment (the simultaneous

rotation of both cameras in opposite directions), allowing

the robot to fixate a visual target with both cameras. The

total weight is 2.4 kg, including the cameras, lenses (0.31 kg

each) and all necessary electronics.

The stereo camera system is mounted on the tilt seg-

ment (2). Two high-resolution 5 megapixel CCD cam-

eras (1) (Basler pilot piA2400-17gm) provide up to 17 fps

each and are triggered by the head’s local controller, pre-

cisely synchronizing the camera pictures with other sensor

data. The wide angle lenses (Schneider Kreuznach Cinegon

1.8/4.8-0902) enable a wide field of view. The mechanical

camera interface simplifies experiments with different types

of cameras. Each camera (1) is positioned with respect

to the receptacle (3) by two locating pins (not shown),

ensuring fast and easy replacement with high repeatability.

The camera stereo basis can be adjusted manually between

133mm and 240mm by interposing camera spacers between

the receptacles (3) and cameras (1). For the experiments

presented here, the stereo basis was set to 200mm.

The pan axis actuator is composed of an AC servo motor

(4) and a harmonic drive gearbox (5) with integrated output

bearing. The tilt axis is driven by a miniature harmonic

drive servo actuator (6). To achieve a compact design, the

actuator is placed below the articulated joint and connected

239

message brokerlocal controllers

low-level

control, I/O

stabilizing

control

trajectory

generation

movement

command server

pose server

vision system

UDP

message

queues

Fig. 3. Basic layout of LOLA’s real-time walking control system.

to the tilt segment (2) by a parallel-crank mechanism (7).

Camera vergence is adjusted by a harmonic drive miniature

servo actuator (8) that connects to the camera receptacles (3)

through a spacial mechanism (9). The four-bar mechanism

is designed for symmetric adjustment of the camera angles

with a single actuator and consists of a double lever arm and

two coupler links of equal length.

C. Computer Architecture

LOLA is controlled by a central control unit (CCU)

mounted on the upper body and nine local controllers carry-

ing out low-level tasks, such as joint control and sensor data

processing. The CCU is based on a PC platform (Core 2Duo

Mobile, 2.33GHz), running the QNX real-time operating

system. The local controllers are a custom development,

since no commercial system was compact enough and pro-

vided all required interfaces.

Trajectory generation and balance control are implemented

on the on-board computer system without any support from

outside except for power supply. An external PC is used

only for monitoring purposes and to give basic operating

commands if the robot is not connected to the vision system.

Due to the high computational cost, vision processing is

performed on an external PC cluster (3 PCs with dual Intel

Xeon Quad-Core each), to which camera data is sent over

two Gigabit Ethernet connections.

IV. WALKING CONTROL

LOLA’s real-time walking control consists of a number of

cooperating processes for tasks such as trajectory generation,

stabilizing control and communication that run on the on-

board computer (see Fig. 3). In the following, the interface to

the computer vision system, control of the viewing direction

and the basic structure of the walking control are presented.

A. Interface to Computer Vision System

Both computer vision system and walking control use local

coordinate systems centered on the robot. However, since

vision and control systems run on different computers and

with different cycle times, it is important to establish a com-

mon frame of reference. Therefore, the walking controller

estimates the position and orientation of its local coordinate

system with respect to a global one using odometry. The

estimate is sent to the vision system together with the current

robot pose (see below). Motion commands are then specified

in this global reference frame. Note that the estimated global

position is only used for communication purposes and does

not have to be exact, since both control and vision processing

are performed in local coordinate frames.

While the vision system needs some information about

the robot’s configuration, the detailed kinematic model with

all DoFs is not required. Therefore, relevant information is

condensed into an abstract robot model with two cameras

and telescopic legs, whose configuration is represented by

five coordinate systems: one for each foot and each camera

and one for the upper body.

The computer vision system specifies the walking speed

and direction by sending the desired average linear and

angular velocity of the robot in the ground plane at a rate of

6Hz. The walking controller uses this input to plan footstep

locations for the following four steps along a circular path.

Currently, the robot does not modify its swing leg position

according to changing walking commands. The viewing

target is specified by a 3-D point in the global coordinate

system. This simple interface enables the robot to smoothly

track static objects while compensating for self-motion and

disturbances.

B. Camera Head Control

For a given kinematic configuration of the robot, the head

pose can be uniquely determined from the 3-D gaze/fixation

point, which is the intersection of the optical axes. The ideal

head joint angles for perfectly tracking the view target are

calculated using the actual robot pose. This leads to auto-

matic inertial stabilization of the cameras and compensation

for self-motion and disturbances.

The trajectory of the desired fixation point can be discon-

tinuous, e.g., when attention switches to a different object.

Therefore, a simple tracking controller would have to use

very low gains to avoid damaging the robot. This, however,

would lead to unacceptable performance when tracking an

object.

To solve this problem, we have implemented a two-part

control strategy that distinguishes between tracking an object

or moving to a new viewing target. This allows us to generate

smooth trajectories for the joint position controller while

achieving good tracking of stationary or smoothly moving

objects. This strategy was designed from a control systems

point of view, but there is a remarkable similarity to eye

movements in biological systems, where there also is a

distinction between volitional saccades on the one hand and

smooth pursuit or optokinesis on the other [20].

While smooth pursuit is often studied in the context of

moving objects, some authors have suggested that it evolved

to keep the point of interest at the center of the retina

during self motion [20], which is also the main purpose of

this control module for our robot. In contrast to biological

240

systems however, our robot uses no direct visual feedback

in the camera control loop.

Switching is based on the current tracking error of the

viewing target: for small tracking errors the smooth pursuit

controller is selected and a saccade is initiated otherwise.

The tracking controller calculates target angles for the joint

position controller by filtering the ideal target trajectories ob-

tained by inverse kinematics. The controller is implemented

as a second order low-pass filter with rate limiter that outputs

desired position, velocity and acceleration.

Trajectories for moving towards a new viewing target

(saccades) are parametrized as 5-th order polynomials. The

six parameters are determined from position, velocity and

acceleration at the current control cycle and at the end of

the trajectory. The final acceleration is set to zero, while the

velocity and position are set to those obtained from inverse

kinematics. The duration of the trajectory is calculated from

the maximal admissible speed for the saccadic motion. Since

the viewing trajectory is recalculated every control cycle, the

viewing target can be switched at any time.

C. Walking Control

The walking controller can be subdivided into reference

trajectory generation and stabilizing control. The trajectory

generator calculates an ideal robot motion in task space

coordinates xd,0 and corresponding contact torques λd,0. The

stabilizing controller then modifies xd,0 and λd,0 to achieve

stable walking. The modified trajectories are mapped to joint

angles that are tracked by a high-gain position controller.

Foot trajectories are planned kinematically using the de-

sired footstep locations. The CoM trajectories are calculated

by solving a two-point boundary value problem for a simpli-

fied three point mass model, where one point is associated

with each foot and the third one with the remaining robot

mass. The equation of motion in the coronal plane is given

by:

mb[zbyb − yb(zb + g)] = −Tx+

ml

∑

i=1..2

[yl,i(zl,i + g)− zl,iyl,i].(1)

Here yi are positions in the lateral direction, zi are vertical

positions, mi are masses and g is the gravitational constant.

The index b marks the body mass point, (l, i) the i-th foot

and Tx is the contact torque. We calculate an ideal contact

torque reference from the planned footstep locations. Since

the foot and zb trajectories are planned independently, (1)

can be solved numerically if either two initial conditions or

two boundary conditions are specified.

To obtain a smooth connection to the current trajectory, we

add an additional constraint for the initial velocity yb(0) to a

normal two-point boundary value problem. Three boundary

conditions require an additional free parameter, which is

provided as a small change to the reference torque. This

planning problem is solved using a collocation method

proposed earlier by one of the authors [21].

Our stabilizing control is based on the observation that

the robot can only influence its overall linear and angular

momentum by changing the contact forces. Based on data

from the inertial measurement unit, the controller calculates

modified contact forces λd,1 and task space coordinates xd,1.

In general, it is not possible to track both position and force

trajectories at the same time. We therefore choose a subset of

contact forces that should be tracked. Currently, we control

the horizontal contact torques and the vertical contact forces

of the foot or feet touching the ground. To track these forces,

a deviation from the reference trajectory is allowed in certain

task space directions. For example, the vertical distance of

the foot from the CoM or the orientation of the foot relative

to the upper body can be modified to control contact torque

and force.

The reference trajectory modification is combined with

a resolved motion rate control scheme to obtain the final

control law:

xd,2 = xd,1 + ST

x∆xλ +Kx

∆t

{

xd,1 + ST

x∆xλ − x}

∆xλ =

∫

∆xλdt (2)

q = (∇qx)#xd,2 − αNN∇qH

Here Sx is the complement of a selection matrix specifying

the position controlled dimensions, Kx is a gain matrix, ∆t

the sampling period and ∆xλ the trajectory modification for

controlling the contact forces. The third equation describes

the mapping from task to joint space using the nullspace

projection matrix N , an objective function H and a gain

αN . Currently, H is used mainly to avoid joint limits and

for returning the robot to a neutral pose during stance.

The same planning and control algorithms were success-

fully applied to our robots JOHNNIE and LOLA. Using

this approach, LOLA reached a maximum walking speed of

3.34 km/h. More details on the stabilizing controller are given

in [22].

V. COMPUTER VISION SYSTEM

We propose a new approach to autonomous navigation

in unknown environments. Similar to a beetle detecting ob-

stacles with its antennae, we exploit the spacial relationship

between the robot and obstacles to find free paths. In fact, our

approach utilizes an analogy that we call tentacle. A tentacle

is a feasible trajectory represented in the local coordinate

system that starts at the current robot position. At every

cycle the system has several tentacles to choose from. Each

of them is rated by the image data. The trajectory with

the highest score is selected for one cycle. At each time

step the robot can choose a new tentacle according to the

new sensor data. The result of our method is an executed

path composed of single tentacle fragments. Our approach

uses a simplified model of feasible paths consisting of 2-D

points in the ground plane. These points roughly correspond

to the robot’s CoM projected onto the ground. The exact

shape of the tentacles can vary according to the motion

capabilities of the robot. We use tentacles with the shape

of circular arcs [13]. The benefit of this approach is the

smoothness of the resulting path. We utilize a set of n = 41

241

Fig. 4. Example for tentacle based navigation: detected contour segmentfeatures are shown in black, projected tentacles in blue and the actuallyselected tentacle in green, surrounded by the corresponding classificationarea shown in red.

precalculated tentacles starting at the robot’s position in

the local coordinate system. Every tentacle has a different

curvature γk resulting in trajectories in different directions.

Curved tentacles tend to leave the field of view, so they are

chosen to be shorter than straight ones.

Since we want a navigation system for obstacle avoidance,

the distance along the tentacle to the first obstacle is a good

basis for evaluation. Not only the point on the tentacle itself,

but also an area around it must to be free (see Fig. 4). The

area within a distance dc is probed for evaluation and is

called classification area (CA).

Our approach does not require any model of the environ-

ment to evaluate the tentacles. This is an important aspect,

as computational time is saved because we do not need

to set up and update a map or solve the SLAM problem.

Instead of probing the tentacle in a map, we project the

classification area onto the image and score the tentacles

according to these image sections. So we simplify the rating

process from a 3-D problem to a 2-D one (Fig. 4), which is

easier to handle and speeds up our approach. At this point

we lose some spatial information of the detected objects.

We would like to come back later to this issue. After we

have defined a classification area in each image, only this

section has to be evaluated for scoring the tentacle. We

therefore extract features that include information about the

presence or absence of obstacles. Since every object has a

contour, we extract pixels with strong edge characteristics

that are then chained to directly connected groups. We use

straight contour segments cs to approximate these point

sequences (see Fig. 4). So one cs is a best fit straight line

through a sequence of points. Straight contour segments are

defined by their start and end point cs = (ps, pe). In order

to select a tentacle k for execution, all tentacles are rated

according to their attractiveness dk and the k = argmaxi diis chosen. Next, we use the set of detected straight contour

segments Sk = {cs1, · · · , csn} within the specific CAk

to evaluate the respective tentacle. If a contour segment is

fractionally located within CAk, only the part within this

area is considered. In the next step the start and end points

of all elements within Sk are mapped to the closest point

on the projected trajectory. This adds up to a set of 2-D

points P 2Dk . We reproject every point in P 2D

k onto the ground

plane to obtain a set of 3-D points P 3Dk . We then estimate

the distance to the robot’s position along the tentacle for

the contour segments of both camera images. The minimum

of all these distances is the score dk for this tentacle. The

proposed approach also works with only one camera, but we

use a second camera for redundancy and to widen the field

of view.

In contrast to [13] and [14] we do not have a 360◦ field

of view. To compensate for this limitation, we select the

most distant point of the last selected tentacle as the fixation

point. As the robot and its camera head move, some tentacles

and the associated classification areas can leave the field

of view. In this case we set dk = 0, which disables the

tentacle. As mentioned above, we lose spatial information by

projecting the tentacles into the camera image. Since there is

no spacial information, we assume that every object is in the

ground plane. Therefore, parts of objects above the ground

are estimated to be farther away than they actually are. If the

robot is directly above the obstacle, the difference between

the estimated and the actual distance is zero. Even if higher

points of an object are estimated to be further away, the

points on the ground are scored correctly. Since we use only

the distance from the nearest obstacle for rating a tentacle,

the errors of the higher points do not affect the correctness

of our approach.

In most cases a weighting function using the distance to

the first object is a good choice for evaluating the tentacles.

In some situations, however, a tentacle can be unsafe even

if dk > 0. For example, stopping the robot is a process,

which requires some time and additional space. We therefore

prescribe a minimal distance dcrash to all obstacles that is

large enough for the robot to safely stop. If a tentacle is

evaluated with a score smaller than dcrash, it is classified as

impassable and dc,k set to zero. If the trajectory is passable,

we simply set dc,k = dk.

Tentacles are used for both perception and execution. After

evaluating the tentacles for both camera images and finding

the tentacle tbest with the highest value dc,k, we set a 2-

D linear velocity (vx, vy) and an angular velocity vφ for

one time step according to tbest. Instead of moving along

the whole trajectory, only the first part is executed. With

the next sensor data set a different tentacle can be selected

and the corresponding instantaneous speed sent to the robot

controller. Sudden and unnecessary changes in the walking

direction are avoided by favoring tentacles similar to the last

one during the scoring process.

The resulting trajectory is composed of different tentacle

segments. Therefore, the robot can walk along arbitrary

smooth paths. If no passable trajectory is found, LOLA starts

turning until a free path is found.

242

(a) LOLA avoids chair

(b) LOLA avoids human being

(c) LOLA avoids scissors

Fig. 5. Experimental Results

VI. EXPERIMENTAL RESULTS

The system was publicly demonstrated more than 25 times

at the Hannover Messe 2010 1. The robot autonomously

explored a 4 × 7m2 wide area while avoiding unknown

objects of various shapes and sizes. We chose objects of

everyday life such as tables and chairs, Fig. 5(a). In order to

show that LOLA does not require any prior knowledge of the

shape, size or color of the obstacles, spectators were asked

for objects to be used as obstacles. The robot successfully

avoided various bags, bottles and even small objects such

as cell phones and scissors, Fig. 5(c). LOLA can not only

navigate in static environments, but also among moving

objects. Our staff members acted as dynamic obstacles by

blocking LOLA’s path, Fig. 5(b).

The experiments showed that our approach enables LOLA

to safely navigate in an unknown environment without

explicitly constructing a model of the environment. Even

1A video can be found at http://www.amm.mw.tum.de/

fileadmin/Image_Archive/Forschung/Robotik_und_

Laufen/Lola_hnf2010_web.mp4

small objects are detected because of the high resolution

cameras. The edge of the carpet has a darker color than

the central area. This contrast is interpreted as a barrier,

preventing LOLA from leaving the carpeted area (see Fig. 5).

Even though we use a simple reactive approach, our method

allows LOLA to handle quite complex tasks such as finding

a way out of a dead-end. This is possible, since LOLA starts

turning around on the spot after reaching the dead-end, until

a feasible tentacle is detected.

VII. CONCLUSION AND FUTURE WORK

We have presented a humanoid robot capable of both

fast walking and autonomous navigation in complex and

unknown environments. The major advantage of our ap-

proach is that it enables navigation among arbitrary obstacles

without requiring color coding or surface texture and that it

does not require an explicit model of the environment.

There are several areas in which we plan to improve and

extend our system in the future. The proposed system will

act as the lowest layer in multi-layered architecture. This

layer can be used in a wide range of scenarios and prevents

collisions in a fast and safe way, but it cannot solve complex

tasks such as climbing stairs, since all objects are classified

as obstacles. If specific objects are recognized, appropriate

actions will be selected by higher layers.

Our next step will be to categorize the perceived objects

into “in the ground plane and passable” (e.g., shadows or rich

texture on the floor) and “not passable”. Furthermore, we

will extend the system so that it can perform more complex

tasks such as stepping onto or over obstacles or climb stairs,

while still safely avoiding unknown objects. This will require

reconstructing specific image areas using stereo vision and

detecting the height of safe footstep locations.

The presented approach selects the tentacle for execution

by evaluating the clearance of each tentacle. This enables

safe navigation, but there is no concept of a goal location.

Combining the clearance function with an objective function

will allow the robot to navigate towards a goal while avoiding

obstacles.

ACKNOWLEDGEMENT

This work is supported by the German Research Founda-

tion (grants UL 105/29 and HU1743/1-1).

REFERENCES

[1] S. Kagami, K. Nishiwaki, J. Kuffner, K. Okada, M. Inaba, and H. In-oue, “Vision-based 2.5D terrain modeling for humanoid locomotion,”in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2003, pp. 2141–6.

[2] K. Sabe, M. Fukuchi, J.-S. Gutmann, T. Ohashi, K. Kawamoto, andT. Yoshigahara, “Obstacle Avoidance and Path Planning for HumanoidRobots using Stereo Vision,” in Proc. IEEE Int. Conf. Robot. Autom.

(ICRA), 2004.[3] K. Okada, S. Kagami, M. Inaba, and H. Inoue, “Plane Segment Finder:

Algorithm, Implementation and Applications,” in Proc. IEEE Int. Conf.

Robot. Autom. (ICRA), 2001, pp. 2120–5.[4] J.-S. Gutmann, M. Fukuchi, and M. Fujita, “A Floor and Obstacle

Height Map for 3D Navigation of a Humanoid Robot,” in Proc. IEEE

Int. Conf. Robot. Autom. (ICRA), 2005.[5] K. Nishiwaki, S. Kagami, J. Kuffner, M. Inaba, and H. Inoue,

“Humanoid “JSK-H7”: Research Platform for Autonomous Behaviorand Whole Body Motion,” in Proc. Int. Workshop Humanoid Human

Friendly Robot. (IARP), 2002, pp. 2–9.

243

[6] S. Kagami, K. Okada, M. Inaba, and H. Inoue, “Design and imple-mentation of on-body real-time depth map generation system,” in Proc.IEEE Int. Conf. Robot. Autom. (ICRA), 2000, pp. 1441–6.

[7] K. Nishiwaki, S. Kagami, J. J. Kuffner, M. Inaba, and H. Inoue,“Online Humanoid Walking Control System and a Moving GoalTracking Experiment,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA),2003.

[8] F. Pfeiffer, K. Loffler, M. Gienger, and H. Ulbrich, “Sensor and ControlAspects of Biped Robot “Johnnie”,” Int. J. Humanoid Robot., vol. 1,no. 3, pp. 481–96, 2004.

[9] R. Cupec, O. Lorch, and G. Schmidt, “Experiments in Vision-GuidedRobot Walking in a Structured Scenario,” in Proc. IEEE Int. Sympo-

sium Ind. Electron., 2005.[10] J.-S. Gutmann, “Multiple plane segmentation with the application

of stair climbing for humanoid robots,” in Symposium on Intelligent

Autonomous Vehicles (IAV), 2004.[11] P. Michel, J. Chestnut, S. Kagami, K. Nishiwaki, J. Kuffner, and

T. Kanade, “GPU-accelerated real-time 3D tracking for humanoidlocomotion and stair climbing,” in Proc. IEEE/RSJ Int. Conf. Intell.

Robots Syst. (IROS), 2007, pp. 463 –9.[12] J. Chestnutt, Y. Takaoka, K. Suga, K. Nishiwaki, J. Kuffner, and

S. Kagami, “Biped Navigation in Rough Environments Using On-Board Sensing,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst.

(IROS), 10-15 2009, pp. 3543 –3548.[13] F. von Hundelshausen, M. Himmelsbach, F. Hecker, A. Mueller, and

H.-J. Wuensche, “Driving with tentacles: Integral structures for sensingand motion,” J. Field Robot., vol. 25, no. 9, pp. 640–73, 2008.

[14] D. Fox, W. Burgard, and S. Thrun, “The dynamic window approach tocollision avoidance,” IEEERobotics & Automation Magazine, vol. 4,no. 1, 1997.

[15] S. Lohmeier, T. Buschmann, and H. Ulbrich, “System Design andControl of Anthropomorphic Walking Robot LOLA,” IEEE/ASME

Trans. Mechatron., vol. 14, no. 6, pp. 658–66, 2009, focused Sectionon Anthropomorphism in Mechatronic Systems.

[16] S. Lohmeier, T. Buschmann, H. Ulbrich, and F. Pfeiffer, “ModularJoint Design for Performance Enhanced Humanoid Robot LOLA,” inProc. IEEE Int. Conf. Robot. Autom. (ICRA), 2006, pp. 88–93.

[17] K. Kaneko, F. Kanehiro, S. Kajita, H. Hirukawa, T. Kawasaki, M. Hi-rata, K. Akachi, and T. Isozumi, “Humanoid Robot HRP-2,” in Proc.

IEEE Int. Conf. Robot. Autom. (ICRA), 2004, pp. 1083–90.[18] K. Kaneko, K. Harada, F. Kanehiro, G. Miyamori, and K. Akachi,

“Humanoid robot HRP-3,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots

Syst. (IROS), 2008, pp. 2471–8.[19] T. Asfour, K. Regenstein, P. Azad, J. Schroder, N. Bierbaum,

A. Vahrenkamp, and R. Dillmann, “ARMAR-III: An Integrated Hu-manoid Platform for Sensory-Motor Control,” in Proc. IEEE-RAS Int.

Conf. Humanoid Robot. (Humanoids), 2006, pp. 169–75.[20] R. J. Leigh and D. S. Zee, The Neurology of Eye Movements. Oxford

University Press, Inc., 2006.[21] T. Buschmann, S. Lohmeier, M. Bachmayer, H. Ulbrich, and F. Pfeif-

fer, “A Collocation Method for Real-Time Walking Pattern Genera-tion,” in Proc. IEEE-RAS Int. Conf. Humanoid Robot. (Humanoids),2007.

[22] T. Buschmann, S. Lohmeier, and H. Ulbrich, “Biped Walking ControlBased on Hybrid Position/Force Control,” in Proc. IEEE/RSJ Int. Conf.Intell. Robots Syst. (IROS), 2009.

244