UvA@Work, Team Description PaperRoCKIn Camp 2015 - Peccioli, Italy

Valerie Scholten, Victor Milewski, Tessa Bouzidi, Celeste Kettlerand Arnoud Visser (a.visser@uva.nl)

Intelligent Robotics Lab, Universiteit van Amsterdam, Science Park 904, Amsterdam, The Netherlands

http://staff.fnwi.uva.nl/a.visser/research/atwork

I. INTRODUCTION

The UvA@Work team consists of four ArtificialIntelligence (AI) bachelor students supported bya senior university staff member. It was foundedin 2013 as part of the Intelligent Robotics Lab1.This initiative strives to involve students of theUniversiteit van Amsterdam (UvA) in robotics. Italso acts as a governing body for all the UvA’srobotic teams such as UvA@Work, The DutchNAO Team [1] (RoboCup Standard PlatformLeague), the Amsterdam Oxford Joint RescueForces [2] (RoboCup Rescue Simulation League)and Dutch UAS (Wildlife Conservation UASChallenge) to guarantee continuation of research,education and competition experience. It alsoenables sharing any gathered knowledge betweenthe different teams.

After its first competition experience with theKUKA youBot at the RoCKIn Camp 2014 in Rome,the UvA@Work team is looking forward to partic-ipate in the RoCKIn Camp 2015 in Peccioli.

1http://www.dutchnaoteam.nl/index.php/irobolab/

II. HARDWARE

The KUKA youBot used by the UvA@Workteam is equipped with Festo FinGrippers and asensor suite consisting of an Asus Xtion Pro Liveto create a disparity map, a Hokuyo UTM-30LXlaser scanner for occupancy grid mapping and aMicrosoft LifeCam HD-5000 for RGB vision. TheAsus Xtion Pro is mounted well above the youBotplatform to allow for a broad view of the localsurroundings. These broad overviews can be pro-cessed as both, RGB and depth images. The HokuyoUTM-30LX laser scanner is mounted in front ofthe youBot platform and provides a high precisiondepth plane over a 270 degree angle. Finally, theMicrosoft LifeCam HD-5000 is mounted on topof the youBot’s end effector to assist pick-up andplacing tasks.

III. SOFTWARE

A. Robot Operating System (ROS)

ROS is a broadly used framework for robotic ap-plication development. It supplies an easily extensi-ble environment of basic components (nodes) whichcan be combined flexibly to form applications. ROSalso provides a parameter server which allows nodes

to use shared parameters. In the current implemen-tation these shared parameters are used to changethe local workings of each node depending on thetask that needs to be carried out. Furthermore, ROScomes with a range of packages, libraries, driversand simulation programs that simplify the use ofstandard platform robots. For the development ofimplementations on the KUKA youBot, the teamof UvA@Work uses the ROS Hydro Medusa ver-sion, published in public repository, as described insection III-G.

B. Image Processing Pipeline

The Image Processing Pipeline (IPP) cur-rently consists of 7 nodes: webcam controller,openni2, image controller, image thresholder, data-matrix detector, tag detector and object detector.The webcam controller takes care of retrievingimage streams from the different RGB camerasmounted on the robot. In the current implementa-tion, only one camera can be used at a time dueto limitations in data capacity of the USB-bus. Towork around this limitation, the active camera canbe assigned depending on the current task by usingROS parameters. For depth cameras the standardROS package Openni2 is used. These two packagessend their images to the image controller node.Depending on the ROS parameters specified, thereceived input is passed on to the right detectorand/or image thresholder node. The detectors areused to locate data matrices, tags (colored sheets)or objects (as specified in the rulebook) in theimage frame. These detectors also rely on the useof the OpenCV2 library. Finally, the detectors sendobtained features to the action controller.By using this pipeline, each node can operate on aseparate thread. Therefore the current implementa-tion is only limited by the image throughput of thecameras used.

C. Manipulation Pipeline

Once localization and navigation tasks are com-pleted, the next step is to fulfil the manipulationchallenge. This could be picking up and preciselyplacing one of the industrial parts of the RoboCupand RoCKIn@Work challenges.

A prerequisite for pick-up is that the transfor-mational function between perception-space andconfiguration-space of the robot arm is known. Toobtain it, the Move-It module of ROS is extendedwith a path-planning algorithm developed by Dorstet al [3], preferable an anytime incremental variantlike [4].

The advantage of using a path-planning algorithmin the configuration space is that evading obstaclesin the environment becomes native in the planning.The challenge with this method is the transfer ofreal-world coordinate points to the configuration-space of the robot. Once the Inverse Kinematics ofthe KUKA youBot is calculated, the obstacles in thereal world can be mapped onto the configuration-space. After this, the path-planning in the configu-ration space becomes a trivial problem that can betackled by several search-algorithms like D*-lite [5]or Dorst’s algorithm [3].

D. Inverse Kinematics

To solve the inverse kinematics, the configurationof the 7 joints of the youBot arm is now defined asa set of homogeneous transformations. Each jointrepresented with a translation matrix and optionallya rotation matrix that represents the rotation ofthe joint over the specified angle θi. The forwardkinematics chain to compute the position of thegripper tip relative to the origin can be calculatedin the following way:

T0 = J1T · J2T · J2R(θ2) · J3T· J3R(θ3) · J4T · J4R(θ4) · J5T· J5R(θ5) · J6T · J6R(θ6) · J7T

The inverse kinematics requires to find the fivejoint angles θi which correspond with a certainlocation T0. The translation and rotation matricesare given in the appendix.

E. Human Robot Interaction (HRI)

In context of the RoCKIn Camp 2014 in Rome,the UvA@Work team started working on a SpokenLanguage Understanding (SLU) application for op-timized customer-robot interaction. As SLU allowsfor a native communication with the robot, HumanRobot Interaction is tailored for real-word use.

Spoken commands are recorded by an attachedmicrophone and sent to the online Google voicerecognition service. The returning result in form ofwritten words is fed to a language model system,tagging words and applying a grammatical struc-ture. As a next step, contained objects and actionsare grounded in reference to a prebuilt databaseof environment and robot. These grounded objectsand actions are than used to create a series ofcommands that can be send to the task managernode within the ROS framework - thus conversedinto a language that is ”understood” by the robot.Tested in RoCKIn Camp Rome were basic direc-tional commands as well as emergency ”STOP”.This SLU interface can also be used as an alterna-tive to the standard command-line @Work interfaceused to specify the task to be carried out.

F. Action Manager

The action manager takes care of all movementrequired by the KUKA youBot. It does so bylistening to the task manager. All actions are par-tially depended on the current state of the robot.Therefore the action manager also keeps an updatedversion of the current joint states.

G. Reusability of the Software

During development of the code, UvA@Workfocused on creating task-specific nodes that can beused as independent modules in the ROS frame-work. On top of that, various top-layer instanceswere created to centrally manage and control themodules. All information streams are assigned tounique default topics that can be subscribed to byother nodes where necessary. By doing so, modu-larity of code and re-usability of modules can beguaranteed.The code of the UvA@Work team is available atgithub2 where the reusable task modules will bemade publicly available, following the initiative ofSwarmLab3.

2https://github.com/IntelligentRoboticsLab/KukaYouBot3https://github.com/swarmlab/swarmlabatwork

IV. RESEARCH AREAS

The University of Amsterdam has been active inthe RoboCup since 1998 in form of varying teamssuch as the Windmill Wanderers [6], ClockworkOrange [7], Dutch Aibo Team [8], the AmsterdamOxford Joint Rescue Forces [9] and the UvA RescueTeam [10].

Since its establishment in September 2013,the UvA@Work team has participated in theRoCKIn@Work Camps 2014 workshop in Rome,Italy [11]. This workshop offered both lectures andpractical sessions on various topics such as naviga-tion, object manipulation and human robot interac-tion. In the latter case, they have been awarded thetitle: “Best practical demonstration on human robotinteraction.”

A. Intelligent House BuildingThe new team members of UvA@Work have

worked with the KUKA youBot in the first yearof their study. They used the youBot to study howdifficult it is to use a robot as construction device[12]. The result of their effort was that they wereable to program the robot to construct two wallsfrom construction material (see Fig. 1) with openloop control.

Figure 1. The result of the construction operation of the KUKAyouBot.

A recording of the construction operation isavailable on YouTube4. As can be seen from this

4https://www.youtube.com/watch?v=M7llxA31NxY

recording, there is some variation in the placementof the material. This variation can only be reducedby introducing a closed loop based on torque orvisual sensors, like the new version of the MicrosoftLifeCam or the Asus Xtion Pro.

V. TASK DESCRIPTION

There are three tasks that can be performed atRoCKIn camp 2015. Out of these three the aim isto do both the tasks named Fill a Box with Partsfor Manual Assembly as well as the task PrepareAssembly Aid Tray for Force Fitting. These tasksare more comparable to our previous work thenthe task about plate drilling. Because the robot isequipped with a Microsoft LifeCam HD-5000, theaim is to recognize the objects and the destinationof the robot by RGB-vision. In a grid, with on oneend the robot and on the other end the destination,there needs to be a search for objects. Hopefullywe will be able to integrate found objects on themap generated by the laser scanner. With the A*-algorithm an optimal path without obstructions canbe found. This way the robot can get to his desti-nation as quick as possible. When it has arrived therobot puts the plate in its rightful place. This canbe done with a computer-vision based approach.

The second task, Prepare Assembly Aid Tray, isan extension of the first task. The first thing toconsider are the paths from the start point to theproducts that need to be assembled. The shortestpath will be the first product our robot is going tofetch. When the robot has arrived and has localizedthe object with RGB-vision, it will pick this productup and put it on his platform. Then the robotcalculates the paths to the rest of the productsand again picks the shortest path. With use of theHokuyo UTM-30LX laser scanner the environmentis constantly scanned so even dynamic obstacles canbe avoided.

VI. CONCLUSION

The RoCKIn Camp, as a weeklong hands-onschool in robotics , will be a great opportunity toboost both the knowledge and experimental skillsof our students. In addition, it will be great to shareideas with teams from other European countries.

The expected result will be a large improvementof the performance of our system to recognize andmanipulate objects in an industrial setting.

REFERENCES

[1] P. de Kok, N. Girardi, A. Gudi, C. Kooijman, G. Metheni-tis, S. Negrijn, N. Steenbergen, D. ten Velthuis, C. Ver-schoor, A. Wiggers, and A. Visser, “Team description forRoboCup 2013 in Eindhoven, the Netherlands,” Dutch NaoTeam, Universiteit van Amsterdam & TU Delft, May 2013.

[2] N. Dijkshoorn, H. Flynn, O. Formsma, S. van Noort, C. vanWeelden, C. Bastiaan, N. Out, O. Zwennes, S. S. Otarola,J. de Hoog, S. Cameron, and A. Visser, “Amsterdam oxfordjoint rescue forces - team description paper - virtual robotcompetition - rescue simulation league - RoboCup 2011,”Universiteit van Amsterdam & Oxford University, July2011.

[3] L. Dorst, I. Mandhyan, and K. Trovato, “The geometricalrepresentation of path planning problems,” Robotics andAutonomous Systems, vol. 7, no. 2, pp. 181–195, 1991.

[4] M. Phillips, A. Dornbush, S. Chitta, and M. Likhachev,“Anytime incremental planning with e-graphs,” in Roboticsand Automation (ICRA), 2013 IEEE International Confer-ence on. IEEE, 2013, pp. 2444–2451.

[5] S. Koenig and M. Likhachev, “D* lite.” in AAAI/IAAI,2002, pp. 476–483.

[6] E. Corten and E. Rondema, “Team description of the wind-mill wanderers,” in Proceedings on the second RoboCupWorkshop, 1998, pp. 347–352.

[7] P. Jonker, B. van Driel, J. Kuznetsov, and B. Terwijn,“Algorithmic foundation of the clockwork orange robotsoccer team,” in Algorithmic Foundations of Robotics VI.Springer, 2005, pp. 17–26.

[8] A. Visser, J. Sturm, P. van Rossum, J. Westra, and T. Bink,“Dutch aibo team: Technical report robocup 2006,” Decem-ber 2006.

[9] V. Spirin, S. Soffia Otarola, and A. Visser, “Amsterdamoxford joint rescue forces - team description paper - virtualrobot competition - rescue simulation league - robocup2014, joao pessoa - brazil,” 2014.

[10] M. Traichioiu and A. Visser, “Uva rescue - team descriptionpaper - agent competition - rescue simulation league - iranopen 2014,” 2014.

[11] S. Negrijn, S. van Schaik, J. Haber, and A. Visser,“UvA@Work - RoCKIn@Work camp 2014 - rome, italy,”Team Description Paper, Intelligent Robotics Lab, Univer-siteit van Amsterdam, The Netherlands, November 2013.

[12] V. Scholten, V. Milewiski, T. Bouzidi, and C. Kettler,“Intelligent house builder,” Project Report, Universiteit vanAmsterdam, June 2014.

APPENDIX: THE TRANSLATION AND ROTATIONMATRICES OF THE KUKA YOUBOT ARM

The forward kinematics chain to compute theposition of the gripper tip relative to the origin canbe calculated in the following way:

T0 = J1T · J2T · J2R(θ2) · J3T· J3R(θ3) · J4T · J4R(θ4) · J5T· J5R(θ5) · J6T · J6R(θ6) · J7T

These are the definitions of each of the matrices.

J1T =

1 0 0 0.1430 1 0 0.00 0 1 0.0460 0 0 1

J2T =

1 0 0 0.0240 1 0 0.00 0 1 0.0960 0 0 1

J2R(θ2) =

cos(170o − θ2) − sin(170o − θ2) 0 0sin(170o − θ2) cos(170o − θ2) 0 0

0 0 10 0 0 1

J3T =

1 0 0 0.0330 1 0 0.00 0 1 0.0190 0 0 1

J3R(θ3) =

cos(θ3 − 65o) 0 sin(θ3 − 65o) 0

0 1 0 0− sin(θ3 − 65o) 0 cos(θ3 − 65o) 0

0 0 0 1

J4T =

1 0 0 00 1 0 0.00 0 1 0.1550 0 0 1

J4R(θ4) =

cos(146o + θ4) 0 sin(146o + θ4) 0

0 1 0 0− sin(146o + θ4) 0 cos(146o + θ4) 0

0 0 0 1

J5T =

1 0 0 00 1 0 0.00 0 1 0.1350 0 0 1

J5R(θ5) =

cos(θ5 − 102.5o) 0 sin(θ5 − 102.5o) 0

0 1 0 0− sin(θ5 − 102.5o) 0 cos(θ5 − 102.5o) 0

0 0 0 1

J6T =

1 0 0 −0.0020 1 0 0.00 0 1 0.1300 0 0 1

J6R(θ6) =

cos(167.5o − θ6) − sin(167.5o − θ6) 0 0sin(167.5o − θ6) cos(167.5o − θ6) 0 0

0 0 10 0 0 1

J7T =

1 0 0 +0.0040 1 0 0.00 0 1 0.0850 0 0 1