Preprint February 14, 2016 – To appear in the Proceedings of the 2016 IEEE International Conferenceon Robotics and Automation (ICRA 2016)

Autonomous Repositioning and Localization of an In situ Fabricator

Timothy Sandy1, Markus Giftthaler1, Kathrin Dorfler2, Matthias Kohler2 and Jonas Buchli1

Abstract— Despite the prevalent use of robotic technologiesin industrial manufacturing, their use on building constructionsites is still very limited. This is mainly due to the unstructurednature of construction sites and the fact that the structuresbuilt must be larger than the machines which build them. Thispaper addresses these difficulties by presenting a repositioningand localization system which, using purely on-board sensing,allows a mobile robot to maintain a high degree of end-effector positioning accuracy while moving among numerousbuilding positions during a building task. We introduce a newlydeveloped machine, called the In situ Fabricator (IF), whosegoal is to bring digital fabrication to the construction site.The capabilities of the repositioning and localization systemis demonstrated through the construction of a vertical stack ofbricks, in which the IF repositions itself after placing each brick.With this experiment, we demonstrate that the IF can build withsub-centimeter accuracy over long building sequences.

I. INTRODUCTION

In comparison to other industries, building constructionexhibits a surprisingly low level of automation. Off-site pre-fabrication, in which smaller segments of a building are madeat a dedicated factory and then brought to the constructionsite, is fairly common today [1]. These methods are limited,however, by the constraints that pre-fabricated parts must betransported to the construction site and that it is difficult toadapt these parts to inaccuracies in the building site and theconstruction process during installation. There has also beensignificant research in the area of large-scale 3D printing andon-site additive manufacturing [2], [3]. These methods arelimited by the fact that the size of the construction machineconstrains the size of the building which it can build andthat current construction methods and environments cannotbe easily adapted to accommodate these processes. For thesereasons, our work focuses on developing machines which canbuild structures directly on construction sites as they existtoday.

The first conceptual work on using mobile robots foron-site building construction dates back to the 1990s [4].While more recent projects [5], have demonstrated the abilityto build non-standard structures and adapt to inaccuracieswithin building processes, these processes typically requiresignificant human intervention to reposition and localize therobot. We believe that in order for a robot to effectively buildstructures at the building scale the machine must be able to

1Timothy Sandy, Markus Giftthaler and Jonas Buchli are with theAgile & Dexterous Robotics Lab at the Institute of Robotics and Intelli-gent Systems, ETH Zurich, Switzerland. {tsandy, mgiftthaler,buchlij}@ethz.ch

2Kathrin Doerfler and Matthias Kohler are with the Chair of Architectureand Digital Fabrication at the Institute of Technology in Architecture, ETHZurich, Switzerland. doerfler@arch.ethz.ch

Fig. 1. The IF placing a brick on top of the stack.

move to new building positions and continue building withminimal setup time and effort. In addition, since constructionsites are constantly changing environments, reliance on exter-nal references and structure within the environment should beminimized. This paper details a system which allows mobilerobots to move to requested building positions and reestablishthe accuracy needed for construction autonomously andusing solely on-board sensing.

This work focuses on a newly developed research platformfor on-site building construction, the In situ Fabricator (IF).In a previous experiment [6] we constructed an undulatedbrick wall using the IF in a semi-autonomous manner. Therobot was driven manually by remote control and the robothad to be closely supervised to ensure building accuracy. Inthis paper we present the next step in the IF’s development.Our main contribution is the demonstration that the IF canexecute an extended building sequence autonomously andusing purely on-board sensing, without exhibiting a degra-dation in accuracy over subsequent repositioning maneuvers.

The remainder of this paper is organized as follows. Wefirst briefly introduce the IF (Sec. II) and the functionalstructure of its repositioning and localization system. Wethen discuss the its key functional sub-systems (Sec. III-V).Next we introduce the Brick Stacking Experiment, whichwas designed to demonstrate the performance of this system(Sec. VI) and present experimental results obtained throughtests performed on the IF in a laboratory environment(Sec. VII). The paper concludes with an outlook to futurework (Sec. VIII).

BASE PLANNER

BASE STATE ESTIMATOR

BASE TRAJECTORY CONTROLLER

SCANNING & REGISTRATION

ARM PLANNING AND CONTROL

Desired Base Trajectory

True robot pose

Estimatedrobot pose

Movement complete

BUILDING SEQUENCER

Desired Base Pose

Fig. 2. Repositioning and localization system diagram

II. THE IN SITU FABRICATOR

The IF is a 1.4 ton mobile manipulator with approximately2.5 m reach which is capable of handling a 40 kg payload.The focus for research on the IF is not to demonstrate thatexisting building processes can be performed by a robot, butto facilitate the development of new building processes whichwere previously unachievable using predominantly manualtechniques. The IF was designed to be versatile and supporta wide range of building tasks.

The main functional requirement of the IF is that it canprovide 1 mm end-effector positioning accuracy with respectto a global reference frame with no loss in accuracy as therobot moves from one building position to the next. Becauseconstruction sites are typically varied and uncontrolled, ourgoal is to minimize the robot’s dependence on existingstructure within its environment. For this reason, the IFwas designed to be a completely self-contained machinewith fully integrated and on-board control, sensing, andpower systems. Importantly, we want to avoid dependenceon external reference systems, so that extensive setup andtedious calibration routines are not required.

To facilitate its design, building, and maintenance, the IFis composed solely of off-the-shelf components. The IF’sarm is a standard industrial robot arm, an ABB IRB 4600.A retrofitted ABB IRC5 arm controller is mounted on-board.The robot’s base is equipped with hydraulically driven tracks.While the combination of electric and hydraulic powersources is not ideal, this was a design compromise madeto reuse existing hardware and minimize robot developmenttime. The system is powered by LI-ion batteries whichprovide energy for 3-4 hours of autonomous operation. Forthe experiments performed in this paper, a vacuum gripperis mounted on the arm end-effector. Because this vacuumsystem is not intended for use on the IF in the long-term it ispowered by mains power, which is the only wired connectionrequired for the robot to execute the tasks described herein.

The control architecture of the IF’s repositioning and local-ization system is shown in Figure 2. The Building Sequencer

manages the overall building task, specifying the sequenceof building positions and the building tasks which should beperformed at each position. It sends a new desired base poseto the Base Planner (described in Section III-A) to initiate arobot repositioning maneuver. Upon successful computationof a collision-free trajectory to the desired position, the BaseTrajectory Controller (Section III-C) steers the robot to thedesired pose, using pose estimates calculated by the BaseState Estimator (Section III-B) in feedback. After completingthe base movement, the current estimate of the base pose isused to calculate the expected position of the structure beingbuilt (from here on referred to as the workpiece). The ArmMotion Planner and Controller (Section IV) subsequentlycomputes and executes the control actions required to scanan area around the workpiece, using a laser range finderthat is mounted on the robot arm end-effector. Point cloudregistration (Section V) is then used to localize the robotwith respect to the workpiece, returning the resulting robotpose to the building sequencer. Building then continues fromthat position and the Building Sequencer can request thenext base position when appropriate. The components of thiscontrol system are discussed in more detail in the followingsections.

III. BASE NAVIGATION

The base navigation sub-system autonomously repositionsthe IF within the construction site. Once the IF completesthe building tasks required in one location (e.g. placesa set of bricks), it moves to the next building position,as requested by the building sequencer. Repositioning isperformed while avoiding known stationary obstacles in therobot’s workspace. The IF also estimates its position whiledriving with sufficient accuracy that it can identify andlocate where it should continue building once it reaches thedesired location. The base navigation sub-system assumesa 2D world. The following section details the trajectoryplanning, robot pose estimation, and trajectory followingcontrol methods used.

A. Base Trajectory Planning

For robot base trajectory planning, we use STOMP [7],which is a stochastic trajectory optimization algorithm. Bygenerating noisy rollouts, it explores the space around aninitial trajectory. The costs of the noisy rollouts are thenused to find an updated trajectory with lower cost.

The base motion planning is based on a planar, wheeleddifferential-drive model as given in [8]. Our cost functionconsists of penalties on the acceleration and jerk of thetrajectory as well as non-smooth obstacle costs. The plannerassumes a static environment and supports obstacles such asa boundary around the area in which the robot is allowed tomove and geometric representations of objects which havealready been constructed. Before planning, the movementduration for the whole base maneuver is estimated from theEuclidean distance and orientation difference between initialand final pose and the maximum admissible track velocity.

In order to speed up the convergence of trajectories withinSTOMP, we found it efficient to run it in two phases. Inphase 1, the planner is initialized with an arbitrary trajectorybetween start and end state, calculated through direct inter-polation. Then, we find an initial feasible solution that avoidsobstacles but does not respect the robot’s non-holonomicconstraints. The second phase uses the solution from phase 1as initial guess but additionally enforces the non-holonomicconstraints of the tracks, which are taken into account aslinear constraints for every sample at every time step. Theexploration noise is conditioned on the linear constraints asoutlined in [9].

B. Base State EstimationDue to the excessive track slip which occurs while a

tracked vehicle drives, conventional odometery is not suffi-cient for tracking the pose of the IF during repositioning. Adedicated state estimator is used instead to estimate the IF’spose in 2D during repositioning. The robot pose is expressedin reference to the workpiece, with the initial pose set by theresult of the previous localization (explained in Section V).Poses are estimated by fusing scans from a laser range finder(Hokuyo UTM-30LX) and gyroscope measurements froman IMU (Xsens MTi-G-700), both mounted rigidly to thebase of the robot, in a “pseudo-maximum-likelihood” mannerusing non-linear least squares optimization with the CeresSolver [10]. The word “pseudo” is used here because theerror function used for scan matching is non-probabilistic.A laser range finder (LRF) is used, as opposed to cameras,because of the low overhead required for scan processingand the potential ability to use the same sensing system toensure robot workspace safety in the future.

Base poses are estimated at the sampling time of the LRF.The latest scan received (referred to as the measurementscan) is fit to a set of earlier scans (called the reference scan)which are spaced over a receding time horizon. Becauseof the robot’s slow driving speeds, scans are treated as arigid set of points obtained at the time of the middle pointin the scan. Scan matching is performed using an entropy-based point cloud quality measure [11]. The error attributedto measurement scan point i is

emi=

N∑j=1

G(pMi − pRj , 2σ2I), (1)

where G(µ,Σ) signifies a Gaussian kernel function centeredabout µ with variance Σ, pM i represents the estimatedposition of the i-th measurement scan point, given the latestestimate of the robot’s current pose, and pRj representsthe j-th reference scan point. Intuitively, this cost functionpenalizes the pair-wise distances between all points in themeasurement and reference scans in a local region whose sizeis defined by σ2. Because measurement and reference scanpoint pairs have only local influence on the error function,kd-tree-based nearest neighbor searching is used to speed uperror function evaluation [12].

Gyroscope measurements are used to improve inferenceon the robot’s orientation. This is necessary because the scan

matching method used is more sensitive to local minima inthe presence of rotational error, as opposed to translationalerror. Gyroscope measurements are forward integrated toestimated orientation of the robot, θk, at the time of anewly received scan. The error attributed to the N gyroscopemeasurements received between the scans taken at times k−1and k is

EkG = θk −[θk−1 +

N∑i=1

ωi∆t]. (2)

The robot pose estimated from the M scan points receivedat time k is

T kWR = argminTkWR

[M∑i=1

emi+ cGE

kG

](3)

and is found using non-linear least squares optimization.The weighting parameter cG determines the influence of thegyroscope measurements on the estimation result.

C. Base Trajectory Controller

Once a trajectory is planned to bring the robot to thedesired position, a simple P-controller is used find the trackvelocities (vl and vr) required to follow the trajectory, usingthe estimated poses described in the previous section infeedback. The resulting track speeds are set as the referencefor the IF’s low-level hydraulic controller. Since drivingspeed is not a significant concern, we represent the desiredtrajectory as a time-independent sequence of 2D robot poses,in a similar manner to [8]. The nominal linear and angularvelocity of the robot (vk and ωk, respectively) at each of theposes is also specified in the desired trajectory.

When an estimated robot pose is received from the stateestimator, the point on the desired trajectory closest to thecurrent pose is found. The nominal linear velocity of thatclosest point is used directly and the controller computes theangular velocity command. The control law at time k is

ωk = ωk + ptekt + pθe

kθ , (4)

where ekt and ekθ are the translational and angular displace-ments between the current robot pose and the closest pointon the trajectory. The commanded track speeds are thencomputed from vk and ωk. The track speeds are limitedfor safety. If the one of the track speeds computed exceedthese limits as the result of a large ωk, the linear velocityof the robot is decreased. This effectively slows down therobot if it gets far from the desired trajectory, allowing itto turn aggressively and get back on track. The controllergains pt and pθ were tuned to balance trajectory followingperformance with smooth robot motions. The performanceof the base navigation sub-system is discussed Section VII.

IV. ARM MOTION PLANNING AND CONTROL

We model the industrial robot arm as a 6-DOF rigid bodysystem with joint state x(t) and joint input torques u(t). Theforward dynamics, inverse dynamics and forward kinematicswere derived using the Robotics Code Generator [13]. Forinverse kinematics we use the Orocos KDL library [14].

For arm motion planning, we replaced the provided pathplanner of the industrial arm by a custom implementationof iLQG [15] in order to obtain more control over armmotions. iLQG is an iterative nonlinear optimal controlalgorithm that calculates a time-varying feedforward controlas well as time-varying state feedback gains. It iterativelylinearizes the system dynamics about the latest estimation ofthe optimal trajectory and designs LQG controllers for eachlocally linearized system. We use a cost function of the form

J = E

h(x(tf )) +

tf−1∑t=0

l(x(t),u(t))

(5)

where the terminal cost h(x(tf )) penalizes deviations fromthe desired joint position and velocity. Our intermediatecost l(x(t),u(t)) is dominated by joint velocity and controleffort penalties. Before planning, the movement duration isestimated using the assumption that it is proportional to theEuclidean norm of the difference between the initial anddesired joint angles. In our case, the algorithm typicallyconverges in less than 5 iterations. Using an efficient, multi-threaded C++ implementation [16], locally optimal armtrajectories are generated in less than 2 seconds.

Although the planning problem considers control inputsin the form of joint torques, we only send the resulting jointposition trajectory to the industrial arm internal controller.The current setup was designed in this way so that it couldsupport the use of dynamic replanning in future work. Notethat this planning setup does not include collision avoidancein calculating arm trajectories.

Whole sequences of arm motion and object manipulationare created by combining the planned trajectories with alibrary of pre-programmed manipulation-primitives, such aspicking up a brick from the brick feeder, and moving a jointat constant velocity.

V. SCANNING AND REGISTRATION

High-accuracy robot localization is achieved by locat-ing the workpiece in point clouds generated using a LRFmounted on the end-effector of the robot arm. After reposi-tioning, the robot scans its environment and the workpieceand ground plane are located in the scan, indicating therobot’s new pose. The following section describes this pro-cess in detail.

A. LRF Scanning

Construction tasks typically afford an abundant amountof prior information about the geometry of a constructionsite and the workpiece, in the form of a CAD model. Forthis reason, the scanning strategy chosen for this applicationmatches point cloud measurements directly to geometricrepresentations of the construction site and workpiece. It isassumed that, in the vicinity of the workpiece, the completegeometric form of the construction site is known. In thisway, a probabilistic LRF sensor model can be used to findthe poses of geometric features in the robot’s environmentwhich maximize the likelihood of acquired point clouds.

Fig. 3. Point cloud of the brick stack, with the registered brick in red.

The geometry of the construction site and workpiece arerepresented as a set of parametrically defined geometricprimitives (e.g. infinite planes and polygonal faces). Prim-itives can either represent objects directly (e.g. an infiniteplane is used to represent the ground of the constructionsite), or they can be grouped into a rigid object (e.g. sixpolygons are grouped to represent one brick). Each object hasan associated pose, which is optimized through registration.In the remainder of this section, the notation TROj is usedto represent the pose of the object associated with geometricprimitive gj in the robot reference frame.

Scans are taken when the robot base is stationary. Scanpoints from the LRF are received in the laser coordinateframe (pL) and poses, representing the pose of the arm end-effector with respect to its base (TRT ), are received from thearm control system. Linear interpolation is used to find thepose of the arm at the time of each scan point measurement.Combined with the pose of the LRF with respect to the armend-effector (TTL), a rigid point cloud of the scan points inthe robot frame are built using:

pRi = TRTi TTL pLi, (6)

where i indicates the i-th scan point. For the remainder ofthis section, all scan points will be expressed in the robotframe and we will drop the R subscript to simplify notation.

To facilitate scanning during sweeping arm motions, laserscan measurements are tightly synchronized in time withthe arm pose measurements. This is done by assigningtimestamps to the scans and arm poses based on the receptiontime of digital synchronization pulses transmitted from eachdevice to the robot’s data acquisition system. Digital pulsetimestamps are matched with their appropriate data packetsby mapping all of the pulse and data packet timestampsto the robot computer’s time frame of reference, usingTICSync [17]. While it is difficult to measure the true timingaccuracy of the system, we have confirmed that there isno visible discrepancy between points clouds produced by

scanning an object at a distance of 10 m with LRF tilt angularvelocities of 3 and 30 ◦ s−1.

B. Object Registration

The registration strategy developed for the IF was designedto take advantage of the non-uniform noise characteristics ofLRFs. At the distances considered for this application (lessthan 2 m), LRFs are typically almost an order of magnitudeless accurate in the direction parallel to the scan ray thanperpendicular to the ray. Specifically, [18] measured that forthe UTM-30LX we are using, at a scanning distance of 2 m,the standard deviation of scan points in the direction of thescan ray is 0.018 m, versus 0.0025 m perpendicular to theray.

Registration is formulated as a non-linear least squaresoptimization problem. An error value is attributed to eachscan point, proportional to the distance from that scan pointto its nearest geometric primitive, and weighted to accountfor the non-uniform noise model of the LRF. The first step incomputing the error assigned to scan point (pi) is to find thegeometric primitive which is closest to that scan point (g∗),given the current set of estimated object poses. The point onthat geometric primitive which is closest to the scan point isthen found and the error is calculated as

ei = (pi − cp(pi, g∗))T Σpi

(pi − cp(pi, g∗)), (7)

where cp(p, g) represents a function which returns the pointon geometric primitive g which is closest to scan point p.Σpi

represents the covariance of the LRF measurements,rotated such that the direction of highest covariance isparallel to the i-th scan ray.

The optimal set of object poses is found by minimizing thesum of squared scan point distance errors. A sufficiently goodinitial guess for the object poses is required to obtain a goodsolution. Generally, the initial guess should ensure that eachobject considered lies close enough to its true location thatsome of the scan points which actually hit that object whenthe measurement was taken are identified as being closest tothat object.

One typical difficulty in dealing with LRF measurementsis the presence of so-called “shadow points”. When a scanray passes closely to the edge of an object, erroneousmeasurements can be generated in which the returned scanpoint lies between the object’s edge and the object behind it.We can easily identify when this is the case, however, sincewe are matching scans directly to the geometry of the scene.When a scan ray lies within some distance of an edge ofa geometric primitive, we flag the scan point as a possibleshadow point, and if the distance between it and the closestgeometric primitive exceeds a threshold, we simply assignzero error to it. Figure 4 shows a point cloud before andafter shadow point removal.

Table I shows the consistency in registration over a set of8 scans of a brick in the same position. Four different armmotions were used to create the scans, each consisting ofa sweep of a different joint of the robot’s arm. All of the

Fig. 4. Point cloud of a wood block on the ground, before (left) and after(right) shadow point removal.

TABLE ICONSISTENCY IN REGISTERED BRICK POSE FOR 8 SCANS TAKEN USING

4 DIFFERENT SCANNING MOTIONS

Std Deviationx 0.44mmy 1.05mmz 0.69mmroll 0.056◦

pitch 0.113◦

yaw 0.717◦

scans were taken at the same distance away from the brickto avoid distortion in the scans as discussed in Section VII.

C. LRF Extrinsic Calibration

The accuracy of the point clouds generated is severelyimpacted by the accuracy of the relative pose of the laserwith respect to the robot’s end-effector in equation 6. Theregistration package developed for the IF was extended tocalibrate this parameter. Although there is not enough spacein this paper to fully evaluate our method with respect toexisting LRF extrinsic calibration methods, the method usedand the results obtained are briefly discussed.

Since the scan point pi in Equation 7 is directly relatedto the extrinsic calibration by Equation 6, we simply needto include the conversion from raw scan points and armposes to scan points in the robot frame in the error functionformulation and we can concurrently solve for the poses ofthe objects within the scene and the extrinsic calibration.Auto-differentiation is used to avoid the analytic computationof what would be complicated Jacobians. With this method,one can find the extrinsic calibration of the LRF by scanningany object of known geometry.

Extrinsic calibration was performed on 9 different data-sets. All of the data-sets are made up of a set of 6 to 8scans taken of a wood cube, of roughly 0.25 m edge length,on top of a flat surface (Figure 4 shows a scan used forcalibration). Each scan was generated by sweeping one ofthe IF’s arm joints in the vicinity of the wood block. Thedata-sets were acquired with the block in different parts ofthe robot’s workspace and in a range of lighting conditions.The standard deviation of the calibration values obtained areshown in Table II. The calibration from each data-set wasobtained by averaging 20 trials with random initial guessesof the extrinsic calibration and the poses of the ground andwood block in the range of ±0.02 m and 3◦. The maximumstandard deviation of the extrinsic calibration from a single

TABLE IILRF EXTRINSIC CALIBRATION CONSISTENCY OVER 9 DATA-SETS

Std Deviationx 2.9mmy 1.4mmz 4.1mmroll 0.389◦

pitch 0.053◦

yaw 0.389◦

Fig. 5. Building sequence for the brick stacking experiment

data-set versus the initial condition was less than 0.5 mmand 0.1◦, showing good robustness versus initial condition.

VI. BRICK STACKING EXPERIMENT

To demonstrate the accuracy and repeatability of the IF’srepositioning and localization system, an experiment wasdesigned in which a vertical stack of bricks is constructedby the robot completely autonomously. After each brickis placed, the robot repositions itself according to a pre-defined building sequence and place another brick fromthat location. While this experiment does not produce astructure which is architecturally significant, it was chosento highlight the mobile autonomy of the robot and providea clear measure of its building accuracy over numerousrepositioning maneuvers.

In this task, the IF places a brick and drives in analternating fashion. The building sequence is defined by asequence of four building positions, expressed with respectto the stack of bricks. A diagram of the building sequenceused is shown in Figure 5. The building positions werechosen to create challenging base repositioning maneuvers(e.g. the robot must change direction two times per looparound the brick stack), and to force the robot to place brickson the stack from different relative positions. The robotmoves between the four building positions sequentially ina loop, creating a repeated hysteresis-shaped pattern aroundthe stack of bricks.

VII. EXPERIMENTAL RESULTS

The IF successfully built a 1.3 m high stack of 26 bricks,over the course of a 25 position building sequence. The

average error in brick position was 4.6 mm in the directionof the long face of the bricks and 3.5 mm in the direction ofthe short face of the bricks. While this error is far from the1 mm accuracy target set for the robot, the IF did successfullydemonstrate its ability to reposition and localize with sub-centimeter accuracy and with high repeatability. A video ofthe brick packing experiment is available online1.

Figure 6 shows the performance of the base navigationsub-system. Shown are the planned and estimated trajectoriesover the course of four base repositioning maneuvers. Atthe four positions indicated with arrows, the discontinuousstep in estimated robot position shows the update in positionresulting from localization with respect to the brick stack. Forthese repositioning maneuvers, the average error between thedesired and estimated (with the Base State Estimator) robotbase poses at the end of the repositioning maneuver was 3 cmand 1.8◦. This shows the accuracy with which the Base Tra-jectory Controller could lead the robot to the desired position.The average change in base pose after localization was 5 cmand 3.5◦. This shows how much the Base State Estimatordrifted on average during each repositioning maneuver. Theresulting base pose after localization was less than 8 cm and4◦ off from the desired pose originally requested by theBuilding Sequencer. This shows the accuracy with which thebase navigation sub-system as a whole brings the robot tothe desired position. These errors were sufficient to allowthe robot to re-locate the workpiece with sub-centimeteraccuracy in all of the 25 maneuvers executed while buildingthe brick stack.

The largest contributing factor to the errors in brickplacement seem to come from an unmodeled behavior ofthe LRF. Up to and during the execution of this experimentalmost all of the bricks placed with noticeable positioningerror were shifted in the direction from which the scan wastaken. This is most likely due to a range measurement biasexhibited by the LRF at short ranges. Figure 7 shows threesets of stationary scans acquired at different heights above thefloor of our lab. While the floor isn’t perfectly flat, the bumpof about 3 cm height measured by the LRF directly belowthe scanner is certainly a superficial artifact. The size of thebump increases as the scanner gets closer to the ground,indicating that bias increases at shorter ranges. This behaviorwas also observed in [18]. It is left as future work to identifyand compensation for this behavior.

VIII. CONCLUSION AND OUTLOOK

In this paper, we have introduced a repositioning andlocalization system for the IF. With this system, the IF hasdemonstrated that it can reposition itself in accordance witha specified building sequence and achieve sub-centimeter ob-ject placement accuracy repeatably and over long building se-quences. The task was performed completely autonomouslyand without dependence on external sensing systems orartificial markers placed within the robot’s environment.

1https://youtu.be/1HPLYOpTYMY

−2 −1 0 1 2

−1

0

1

x [m]

y[m

]Estimated PositionPlanned Position

Fig. 6. Planned and estimated robot base trajectories for four repositioningmaneuvers. The cross shows the position of the brick stack. The red arrowsindicate where the robot stopped and placed a brick.

−3 −2 −1 0 1 2 3

−2

0

2

4

·10−2

x [m]

y[m

]

0.4 m height1.1 m height2.4 m height

Fig. 7. Stationary LRF scans taken at three heights from the ground. Thelarge bump in the points directly in front of the laser (at x = 0) indicatesa bias of the LRF at short ranges.

As a next step, we intend to integrate this system into alarge-scale building process and demonstrate it in a morerealistic construction site environment. While we have iden-tified one area in which the LRF-based localization systemcan be improved, we also plan to explore different sensingmodalities, such as stereo vision, which will allow for real-time feedback during manipulation tasks. In this way, wehope to get closer to reaching our target of 1 mm accuracy.Beyond the IF sensing system, future work is planned inimproving the system’s planning and controls capabilities.In this work, we deliberately avoided motions that wouldbring the system to the boarders of its dynamic stabilitylimits. Therefore, we particularly intend to push the motionplanning and control capabilities beyond the currently usedseparate arm and base planning, in order to be able to executedynamic whole-body maneuvers.

ACKNOWLEDGEMENTS

This research was supported by Swiss National Sci-ence Foundation through the NCCR Digital Fabrication(NCCR Digital Fabrication Agreement #51NF40 141853)and a Professorship Award to Jonas Buchli (AgreementPP00P2 138920). The authors would like to thank MichaelNeunert and Farbod Farshidian for providing their imple-mentation of the iLQG algorithm.

REFERENCES

[1] U. Knaack, S. Chung-Klatte, and R. Hasselbach, Prefabricated Sys-tems: Principles of Construction. Springer Verlag NY, 2012.

[2] B. Khoshnevis, “Automated construction by contour craftingrelatedrobotics and information technologies,” Automation in construction,vol. 13, no. 1, pp. 5–19, 2004.

[3] P. Bosscher, R. L. Williams, L. S. Bryson, and D. Castro-Lacouture,“Cable-suspended robotic contour crafting system,” Automation inConstruction, vol. 17, no. 1, pp. 45–55, 2007.

[4] J. Andres, T. Bock, and F. Gebhart, “First Results of the Developmentof the Masonry Robot System ROCCO,” in Proceedings of the 11thISARC in Brighton (International Symposium on Automation andRobotics in Construction), pp. 87–93, 1994.

[5] V. Helm, S. Ercan, F. Gramazio, and M. Kohler, “Mobile roboticfabrication on construction sites: Dimrob,” in Intelligent Robots andSystems, 2012 IEEE/RSJ International Conference on, Oct 2012.

[6] K. Dorfler, T. Sandy, M. Giftthaler, F. Gramazio, M. Kohler, andJ. Buchli, “Mobile Robotic Brickwork - Automation of a DiscreteRobotic Fabrication Process Using an Autonomous Mobile Robot,” inRobotic Fabrication in Architecture, Art and Design, 2016.

[7] M. Kalakrishnan, S. Chitta, E. Theodorou, P. Pastor, and S. Schaal,“Stomp: Stochastic trajectory optimization for motion planning,” inRobotics and Automation (ICRA), 2011 IEEE International Conferenceon, pp. 4569–4574, May 2011.

[8] N. Sarkar, X. Yun, and V. Kumar, “Dynamic path following: a newcontrol algorithm for mobile robots,” in Decision and Control, 1993.,Proceedings of the 32nd IEEE Conference on, Dec 1993.

[9] M. Kalakrishnan, A. Herzog, L. Righetti, and S. Schaal, “MarkovRandom Fields for Stochastic Trajectory Optimization and Learningwith Constraints,” in Proceedings of Robotics: Science and Systems,2013.

[10] S. Agarwal, K. Mierle, et al., “Ceres solver.” http://ceres-solver.org.[11] M. Sheehan, A. Harrison, and P. Newman, “Self-calibration for a 3d

laser,” The International Journal of Robotics Research, 2011.[12] J. Elseberg, S. Magnenat, R. Siegwart, and A. Nuchter, “Comparison

of nearest-neighbor-search strategies and implementations for efficientshape registration,” Journal of Software Engineering for Robotics,2012.

[13] M. Frigerio, J. Buchli, and D. Caldwell, “Code generation of algebraicquantities for robot controllers,” in Intelligent Robots and Systems(IROS), 2012 IEEE/RSJ International Conference on, Oct 2012.

[14] R. Smits, “KDL: Kinematics and Dynamics Library.”http://www.orocos.org/kdl.

[15] E. Todorov and W. Li, “A generalized iterative LQG method forlocally-optimal feedback control of constrained nonlinear stochasticsystems,” in Proceedings of American Control Conference, IEEE,2005.

[16] M. Neunert, F. Farshidian, and J. Buchli, “Adaptive Real-time Non-linear Model Predictive Motion Control,” in IROS 2014 Workshop onMachine Learning in Planning and Control of Robot Motion, 2014.

[17] A. Harrison and P. Newman, “Ticsync: Knowing when things hap-pened,” in Robotics and Automation (ICRA), 2011 IEEE InternationalConference on, pp. 356–363, May 2011.

[18] F. Pomerleau, A. Breitenmoser, M. Liu, F. Colas, and R. Siegwart,“Noise characterization of depth sensors for surface inspections,”in Applied Robotics for the Power Industry (CARPI), 2012 2ndInternational Conference on, pp. 16–21, IEEE, 2012.