Robotica (2009) volume 27, pp. 459–468. © 2008 Cambridge University Pressdoi:10.1017/S026357470800489X Printed in the United Kingdom

A CORBA-based simulation and control frameworkfor mobile robotsZhang Zhen∗, Cao Qixin, Charles Lo and Zhang LeiResearch Institute of Robotics, School of Mechanical Engineering, Shanghai Jiao Tong University,Shanghai, 200240, P.R. China.

(Received in Final Form: June 15, 2008. First published online: August 12, 2008)

SUMMARYThis paper presents a distributed multiple mobile robotsframework which allows programming and control ofvirtual and real mobile robots. The system provides the mapbuilding, path planning, robot task planning, simulation,and actual robot control functions in an indoor environment.Users can program the virtual robots in a customizedsimulation environment and check the performance ofexecution, i.e., if the simulation result is satisfying, userscan download the code to a real robot. The paper focuses onthe distributed architecture and key technologies of virtualrobots simulation and control of real robots. A methodfor construction and transfer of a key index value (whichstores the robot configuration) is proposed. Using thismethod, only the robot key configuration index is needed tobuild the robot in the virtual environment. This results inreduced network load and improved real time performanceof the distributed system. Experiments were conducted tocompare the performance of the proposed system with theperformance of a centralized system. The results show thatthe distributed system uses less system resources and hasbetter real time performance. What is more, this frameworkhas been applied to Yaskawa’s robot “SmartPal.” Thesimulation and experiment results show that our roboticframework can simulate and control the robot to performcomplex tasks.

KEYWORDS: Mobile robot; CORBA; Simulation; Realrobot control.

1. IntroductionIn recent years there is an increase in the use of robots in fieldsof replacing humans to fulfill complex and dangerous tasks.The robotic system must be reliable and coordinated whilefinishing different subtasks such as perception, planning,and navigation. Thus some robotic platforms have beendeveloped to test the control algorithms and to evaluate therobot performance.

Currently, we at Research Institute of Robotics in ShanghaiJiao Tong University are developing a robotic simulationand control framework in collaboration with robot producer

* Corresponding author. E-mail: zzh2200 0@126.com, zzh2000@sjtu.edu.cn

Yaskawa Electric Corporation, Japan, entitled “Simulation ofMobile Robots Navigation (SMRN),” based on their interestin distributed precise simulation and control for mobile robotSmartPal.1 This project was initiated and a new frameworkproposed due to the lack of specific functionalities whichare prerequisites specified by Yaskawa for their SmartPalrobots, such as, support for multiplatforms (OS) or systemportability, 2-D and 3-D generator and simulator, etc., whichare not currently supported in other simulation frameworks.These other frameworks will be briefly described in the nextsection.

Our previous work realizes a centralized simulation andcontrol system. The users can easily program a dual-armmobile robot, preview, and check the robot motion in a 3-Dsimulation environment.2 However, due to a need for a largeamount of precise calculations that need to be performed, thesystem uses a considerable amount of computing resources,which makes it difficult for it to be used on a normal PCfor multiple mobile robot simulations. Thus, a distributedflexible navigation simulation system based on CORBA3–5

was developed. We have also implemented our systeminto SmartPal robots. There are currently no other existingCORBA systems which provide such a complete set ofservices for mobile service robots.

Our system can be divided into several components: aMapEditor server, a simulation server, and robot clients. Theyare connected via the CORBA bus and can be deployedin different PCs with different operating systems, whichextends the portability of the system. The system providesmap building, path planning, simulation, and actual robotcontrol services. A robot “key index” is proposed to describethe robot’s model. Each robot’s kinematics motion can becalculated in different client modules separately. Each clientprocesses computationally expensive tasks such as calculat-ing the related robot’s kinematics motion according to its keyindex value and rebuilding the robot and its local environmentscene. Other key technologies, such as multirobot simulationand control mechanism, and seamless migration betweensimulated and actual robots are also proposed.

The remainder of the paper is organized as follows:Section 2 introduces the related works. Section 3 presentsour distributed simulation and control simulation systemarchitecture. Section 4 introduces the related mobile robotSmartPal. The proposed key technologies are presented inSection 5. Experiments to test the distributed system areshown in Section 6. And the conclusion is given in Section 7.

460 CORBA-based simulation and control framework for mobile robots

2. Related WorksWith the rapid progress in computer and communicationtechnology, robotic systems are fast becoming larger andmore complicated. Therefore, a framework is required thatcan integrate reusable components for which various com-panies and individuals contribute their technologies. Manyresearchers have proposed and implemented their solutionsrespectively. ORiN (Open Resource interface for the Net-work/Open Robot interface for the Network) is a middlewareframework, which offers the standard communication inter-face over various FA (factory automation) equipment includ-ing a robot, but it is mainly developed for industrial robotsin some structural environments.6,7 Orocos is a free softwareproject that includes a set of class libraries and applicationframework, and a hard-real-time kernel for all possiblefeedback control applications.8,9 Toshiba has proposed theopen robot controller architecture (ORCA) based on the robottechnology (RT) reference model proposed by Toshiba, so asto allow RT components to be easily packaged. ORCA usesdistributed object technology to enable such components tobe used transparently anywhere via the network.10,11 SONYis actively promoting OPEN-R, which involves the use ofmodular hardware components, such as appendages that canbe easily removed and replaced to change the shape andfunction of the robots, and modular software components thatcan be interchanged to change their behavior and movementpatterns. However it is only developed for SONY’s four-legged entertainment robot prototype.12

BREVE is a simulation environment meant for thedevelopment of artificial life in a physically simulatedworld. It uses a scripting language that allows controlstrategies and event-based reactions to the environmentfor large numbers of agents.13 CARMEN14 uses themiddleware framework MARIE (Mobile and AutonomousRobot Integrated Environment)15 to build the mobile robotcontrol and simulation system. However, there is still not anintegrated platform that supports customized environmentmodeling, graphical programming, virtual robots simulation,

and real robots control functions. Some platforms, suchas Player/Stage/Gazebo, provide environment modeling,simulation, and real robot control, but it can only be usedon a Linux operating system.16,17 The Microsoft R© RoboticsStudio is a Windows-based environment for hobbyist,academic, and commercial developers to create roboticapplications for a variety of hardware platforms. It includesa lightweight REST-style, service-oriented runtime, a set ofvisual authoring and simulation tools, as well as tutorialsand sample code to help users get started. However, all ofthese systems are mainly developed for general robots, soit is difficult to realize an appropriate simulation and realcontrol for redundant dual-arm mobile robots.

3. System ArchitectureThe architecture of the distributed system is presented inFig. 1. It comprises three components: a MapEditor server,a simulation server, and robot clients. The MapEditor serveris responsible for an indoor simulation environment modelbuilding and path planning services. The simulation serverprovides the virtual sensor and simulation services to allowobstacle avoidance. Its Omni database keeps records of allthe information of the simulation. The robot client invokesthe methods from the CORBA server and presents the 3-D simulation result to the users. These functional partscommunicate via the CORBA bus.

As a result of using CORBA, the service implementedobject in these servers can be remotely and transparentlyinvoked from the clients regardless of their hardware,operating systems, and programming language. All theservers and clients can be distributed on different computersusing CORBA middleware, the JavaTM IDL developed bySun Microsystems.18 JavaTM IDL is freely available and isa fully compliant implementation of the CORBA standard. Itprovides interoperability between applications on differentmachines in heterogeneous distributed environments and

Fig. 1. System architecture.

CORBA-based simulation and control framework for mobile robots 461

Fig. 2. Two maps in a MapEditor: (a) geometrical map, (b) topological map.

seamlessly interconnects multiple-object systems. Eachcomponent of the system is presented as follows.

3.1. MapEditor server componentThe MapEditor server is used as a unique virtual environmentfor multiple mobile robots to work in. The server contains twomodules: a map building module and a path planning module.The user can build a customized environment according toa real world using a map building module, save the objects’shapes, positions, and other geometrical data in a geometricalmap, and save the path nodes in a topological map as well(Fig. 2).

The path planning module is used to determine a feasiblepath between the start and the end points specified by the user.Figure 3 shows a communication example between the robotclient and the path planning module. In Fig. 3(a), the clientspecifies the start point and the end point, calls the “getPath”method using the CORBA interface, and receives a set of path

points from the path planning module. Figure 3(b) showsthe map for topological information. Figure 3(c) shows theformat of path points returned by the server.

3.2. Simulation server componentIn the simulation server, the communication managementmodule is responsible for recording all the registeredinformation from the clients and to realize a simple loadbalance. All robot clients that want to join the simulationenvironment must first register with this module. Themanagement module accepts the client’s request and putsthe client’s name in the register list. The robot client can callon the required methods using the CORBA interface. If thenumber of clients reaches the upper limit of the permissionsallowed, the communication management module willdisallow other clients from logging on to the server.

All the simulation data is stored in the Omni database.It contains an elevators data list and a floor data list which

Fig. 3. Path planning communication: (a) communication process, (b) metric and topological identifier, (c) Data structure.

462 CORBA-based simulation and control framework for mobile robots

records multiple floor information of a whole building. Thisdata structure makes it easier to exchange the single floor’sdata between the Omni database (simulator server) and thedata pool of a robot client.

The 2-D simulator server loads the data from the Omnidatabase and draws the simulation scene in a 2-D image.This tool has the capability of displaying the simulation sceneand to allow the user to monitor the simulation in the wholebuilding. In the monitor panel, users can observe differentfloor scenes, see the robots’ positions and velocities, andreceive the data from the virtual sensors.

The virtual sensor module provides virtual laser rangefinder service and proximity sensors service. It receives therobot client’s request with parameters such as the position ofthe robot and the position and direction of the virtual sensor,and returns laser scan angles and distance information. Theinformation is similar to the information returned by a realsensor but it does not take into account the sensor error andthe noise. Usually, a virtual robot just follows a trajectorypredefined by a graphic programming module, and the virtualsensor data are used by the virtual robot to detect the dynamicobstacles which are not predefined. So the virtual robot cantake into account the obstacles present and feed this data intothe path planning module and obtain a new feasible path.

3.3. Robot client componentThe robot client is a development module for users toprogram, control, and observe a virtual robot in a simulatedenvironment or an actual robot in the real world. It isstructured in the CORBA client that calls methods fromdifferent server components. One robot client componentstands for one virtual or actual robot. The modules in therobot include the following:• Graphic programming module (GPM): It is used to specify

the robot’s task by using a list of motion icons (Fig. 4). Theuser can edit the robot key index values in the teaching

box to define a motion and check the result in its 3-Dviewer. Once an icon is programmed, it can be saved intoan icon list. Using this module the user can program therobot to complete the motion tasks.• Control Module: There are two modes: virtual robot modeand the actual robot mode. In the virtual mode, the modulecalls the SmartPal robot’s virtual “Kinematics Engine”19

to calculate each linkage position, and returns the resultsto the GPM. Then the GPM sends the robot’s data to thedata pool for 3-D simulation in the robot client. In theactual robot mode, the module can download the codes toan actual robot to execute the related task.• Data Pool: It is used to save simulation data related tothe current robot. For example, if a virtual robot is onthe second floor in a building simulation environmentwith multiple floors, its data pool only keeps the objects’information on the second floor.• 3-D Simulator & Monitor: It loads the simulation datafrom the data pool, and constructs the robot in a 3-D virtualenvironment. When the simulation is verified, users candownload the codes to a real robot for execution.

4. Mobile Robot SmartPalThe system which is presented in this paper has been realizedon the dual-arm mobile robot SmartPal. So far it is mainlyused as a service robot. Figure 5 shows a SmartPal used forexperiments in an indoor environment. It is equipped with apair of 7-DoF (degrees of freedom) arms and grippers. Anomni-directional wheel platform is used for planar motion.The sensors equipped in the robot include a laser range sensorin the waist and eight proximity sensors mounted around theomni-directional wheel platform. The Liquid Crystal Displayand touch panel are attached to the front of the robot’supper body. They are used for displaying the robotic currentinternal state (e.g., working, waiting, exceptional) or for userinteraction (e.g., choosing the work model).

Fig. 4. The graphic programming module.

CORBA-based simulation and control framework for mobile robots 463

Fig. 5. SmartPal robot.

5. Key Technologies in the Distributed Simulationand Control

5.1. Robot’s key index definition and constructionThe virtual robot is constructed using Java3D. The virtualrobot comprises several basic parts which are described byVRML (Virtual Reality Modeling Language). To assemble aSmartPal robot, the basic parts are defined as a Branch Groupand the robot joints are defined as a Transform Group. Thesejoints and parts of a SmartPal robot, shown in Fig. 6, aredescribed in Java3D as a node chain.

We define the robot joints’ angles, positions, and thedistance values in x and y directions as the key index. Onlykey index is needed to construct the robot.

In order to realize the key index transfer in the distributedsystem, we define the robot’s key index data structure in theCORBA interface. Its UML is shown in Fig. 7.

Users can invoke the methods “send KeyIndex()” and“receive KeyIndex()” to transfer and share multiple robots’key indexes. Once the key index is obtained, we use itto rebuild the related robot and control the virtual or realrobot. Figure 8 displays the coordinate frame chain, rotationdirections, and angles of robot rotation. In this way, theposition of each joint in robot structure can be described.

5.2. Multirobot control and simulation mechanismMultithreading in Java is used to realize the multirobotcontrol or simulation. Two threads are set up: a manipulation

Fig. 6. Architecture of robot basic nodes.

464 CORBA-based simulation and control framework for mobile robots

Fig. 7. The UML of CORBA interface.

Fig. 8. Coordinate chain on robot rods.

thread and a detection thread. In order to improve the realtime performance of the system, a data pool is created in theclient to store the current local environment and robots’ stateinformation. The robot client need not always invoke thewhole Omni database information on the server side, but justcall the local environment data from its own data pool. Thethreads mentioned access the data pool alternately. Thereis a “supply and demand” relationship between them. Aspresented in Fig. 9, the manipulation thread calls the controlmodule to obtain the current robot information and updatesthe data in the data pool. The data pool also exchanges datawith the Omni database. For more efficiency, if a robot ispositioned on a certain floor, in the client’s data pool, onlythe data of the robots and objects on that specific floor isloaded. The detection thread checks the data pool constantlyand, as soon as there is a change in the data pool, the robotis rebuilt in the 3-D simulator and monitor module. Thismechanism is implemented for all the robots in a multirobotsystem.

Fig. 9. Multirobot simulation mechanism.

5.3. Seamless migration between simulated and actual robotsThis system can be applied to both virtual and physicalrobots. If the program for a new job is satisfied in simulation,

CORBA-based simulation and control framework for mobile robots 465

Fig. 10. The architecture of migration between simulated and actualrobots.

the code can be downloaded to real robots. The relatedarchitecture is shown in Fig. 10.

The control component uses a model switcher to set eithera simulated or a real robot control model. If the switcher isin “simulated model,” the high controller will use a “robotsimulation adapter” to invoke the “virtual controller API”which reads the robot’s current status from the “KinematicsEngine,”19 which is provided by Yaskawa, and return tothe graphic programming environment. If the switcher isin “Control Model,” the high controller will use a “RobotHardware Adapter” to invoke the “I/O library API” to controlthe hardware.

6. Experiments and ResultsExperiments were performed to compare the system loadbetween the centralized system2 and the distributed systemwhich is used for controlling actual robots or simulation.

Each computer in the experiments had a Celeron (R)CPU 2.40 GHz and 1230 M usable memory (512 M physicalmemory and 718 M virtual memory).

First, the centralized system was used to perform themultiple-robot simulation. Figure 11 presents the utilizationrate of the CPU (Fig. 11(a)) and the memory (Fig. 11(b))in the centralized system. The utilization rate of both theresources increased linearly with the number of robots in thesimulation. The experiment with four or more robots faileddue to lack of memory.

In the distributed system, the MapEditor server and thesimulation server were installed on a server computer. Fivevirtual robot clients were created to work together in avirtual floor environment. In order to show the CORBA-based system’s independence, the MapEditor server andsimulation server ran on Microsoft Windows XP Professionalwith Service Pack 2, and the other robot clients ran onLinux (Ubuntu 6.10 kernel 2.6.17-11-generic). Each robotexecuted its motion list separately and followed a predefinedtrajectory. Each robot’s kinematics status was calculated viathe related motion module in each robot client and sent tothe simulation server. The simulation is shown in Fig. 12.Because there were no dynamic obstacles predefined in this

Fig. 11. Centralized system load: (a) CPU utilization, (b) memory utilization.

Fig. 12. The scene of the simulation.

466 CORBA-based simulation and control framework for mobile robots

Fig. 13. Distributed system load: (a) server computer CPU utilization, (b) server computer memory utilization.

experiment, the virtual sensors were not used. The computerswere interconnected via a fast Ethernet (10 Base T). Fig-ure 13 presents the rate of the CPU and the memoryutilization with an increase in the number of connectedclients. Both rates of increase in the CPU usage and memoryusage of the distributed systems were less than in the caseof the centralized system. In this case that three clientsconnected to the server computer, for a three-robot simulation(Fig. 13(b)), shows that the server computer only spends63 MB of memory, and the CPU average utilization is 26.58%(Fig. 13(a)). The maximum CPU usage is 43.8% which is lessthan in the case of the centralized system.

The advantage brought about by the distributed systemdemanded less CPU and memory usage. However, itsshortcoming is that the clients need a longer response timebetween the start of a method invocation and the arrival ofthe returned data. Figure 14(a) presents the instantaneousresponse time recorded over 20 s of simulation. The averageresponse time value increased with the number of robots inthe simulation, as shown in Fig. 14(b). In the case where fiveclients called the service from the server, the maximum ofinstantaneous response time is less than 100 ms, which meetsthe requirements of a mobile robot simulation.

Finally, the code of robot I is downloaded to a real robot. Itis designed to handle a task of handing over a piece of paper.The code is composed of a sequence of subtasks: like “moveto the desk A,” “pick up the paper,” “turn back,” “move to doorB,” and so on. Figure 15 displays the screens of the simulation

and the actual robot. It shows that the proposed system cancontrol the robot SmartPal to finish complex tasks.

7. ConclusionsIn this paper, a CORBA-based distributed programmingsystem is proposed to realize the control and simulation ofmultiple mobile robots. In comparison to some non-CORBAsystems, such as Player/Stage/Gazebo, Microsoft RoboticsStudio, and so on, our system facilitates interoperability ofdifferent operating systems (the experiment in Section 5executed in Windows and Linux OS has demonstrated thisfeature). In comparison to some CORBA systems, such asORCA, BREVE, CARMEN, and so on, our system providesthe robot’s map building, path planning, graphic motionplanning, simulation, and actual robot control function.Users can program virtual robots in a customized simulationenvironment, and check the executing performance; if thesimulation result is satisfying, users can download the code toa real robot to execute. The robot key index for configurationis proposed and transferred between different robotic clients,and the robots are built in a virtual environment. This resultsreduced network load and improved real time performance ofthe distributed system. What is more, we have implementedour system to Yaskawa’s “SmartPal” robot. The simulationand experiment results show that our robotic framework cansimulate and control the robot to perform some typical dailytasks such as picking and placing everyday objects (e.g.,

Fig. 14. Response time in distributed system communication: (a) instantaneous response time, (b) average response time.

CORBA-based simulation and control framework for mobile robots 467

Fig. 15. Seamless migration between a simulated and real robot.

cups, glasses, bottles, plates, and cutlery) as well as operatingswitches (e.g., light, coffee machine, and cooker) and handles(e.g., doors, drawers, and refrigerators). It is expected thatthis framework will significantly aid in the development ofdual-arm mobile robots in the home environment.

AcknowledgmentsThis work was supported in part by the National HighTechnology Research and Development Program of Chinaunder Grants 2006AA04Z261 and 2007AA041703, andsupported in part by the National Natural Research underGrant 50705054. The authors gratefully acknowledge thesupport from Yaskawa Electric Corporation for supportingthe collaborative research funds and the SmartPal robot. Theyalso thank Mr. Ikuo Nagamatsu and Mr. Kazuhiko Yokoyamaat Yaskawa for their cooperation.

References1. K. Matsukuma, H. Handa and K. Yokoyama, “Vision-

Based Manipulation System for Autonomous Mobile Robot‘Smartpal”’. Proceedings of the Japan Robot AssociationConference, Yaskawa Electric Corporation, Japan (Sep. 2004).

2. Qiu Chang-wu, Cao Qi-xin, Ikuo Nagamatsu and KazuhikoYokoyama, “Graphical programming and 3-D simulationenvironment for Robot,” Robot 27(5), 436–440 (Sep. 2005).

3. Object Management Group. White paper on benchmarking,Version 1.0, OMG document bench/99-12-01 (1999).

4. M. Henning and S. Vinoski, Advanced CORBA Programmingwith C++ (Addison Wesley, Reading MA, 1999).

5. Object Management Group. OMG Robotics DomainSpecial Interesting Group (DSIG) Homepage. Available:http://robotics.omg.org.

6. M. Mizukawa, H. Matsuka, T. Koyama, T. Inukai, A. Noda,H. Tezuka, Y. Noguchi and N. Otera, “ORiN Open RobotInterface for the Network – The Standard Network Interfacefor Industrial Robots and its Applications,” InternationalSymposium on Robotics Stockholm (ISR2002), No.45 (Oct.2002).

7. M. Mizukawa, H. Matsuka, T. Koyama, T. Inukai, A. Noda,H. Tezuka, Y. Noguchi and N. Otera, “ORiN: Open RobotInterface for the Network – The Standard and Unified NetworkInterface for Industrial Robot Applications,” SICE AnnualConference, Osaka (2002), pp. 1160–1163.

8. Orocos: Open Robot Control Software. http://www.orocos.org.9. C. Schlegel and R. Worz, “The Software Framework SmartSoft

for Implementing Sensorimotor Systems,” IEEE/RSJ Interna-tional Conference on Intelligent Robots and Systems, IROS’99, Kyongju, Korea (Oct. 1999) pp. 1610–1616.

10. Fumio Ozaki, “Open Robot Controller Architecture (ORCA),”Proceedings of IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS2004), Workshop onRobot Middleware toward Standards, Sendai, Japan (Sep.2004).

11. Fumio Ozaki, “Open Robot Controller Architecture (ORCA),”Advanced Intelligent Mechatronics (AIM2003) Workshop:Middleware Technology for Open Robot Architecture, Kobe,Japan (Jul. 2003).

12. Kohtaro Sabe, “Open-R: An Open Architecture for RobotEntertainment,” IEEE/ASME International Conference onAdvanced Intelligent Mechatronics (AIM2003) Workshop:Middleware Technology for Open Robot Architecture, Kobe,Japan (Jul. 2003).

13. J. Klein, “BREVE: A 3-D Environment for the Simulationof Decentralized Systems and Artificial Life,” Proceedingsof Artificial Life VIII, 8th International Conference on the

468 CORBA-based simulation and control framework for mobile robots

Simulation and Synthesis of Living Systems (MIT Press, 2002)pp. 329–334.

14. M. Montemerlo, N. Roy and S. Thrun, “Perspectives onStandardization in Mobile Robot Programming: The CarnegieMellon Navigation (CARMEN) Toolkit,” Proceedings ofIEEE/RSJ International Conference on Intelligent Robots andSystems, Las Vegas (2003) pp. 2436–2441.

15. C. Cote, Y. Brosseau, D. Letourneau, C. Raıevsky and F.Michaud, “Robotic software integration using MARIE,” Int.J. Adv. Robot. Syst. – (Special Issue on Software Developmentand Integration in Robotics) 3(1), 55–60 (2006).

16. B. P. Gerkey, R. T. Vaughan and A. Howard, “The Player/StageProject: Tools for Multi-Robot and Distributed SensorSystems,” Proceedings of the International Conference on

Advanced Robotics (ICAR 2003), Coimbra, Portugal (Jun. 30–Jul. 3, 2003) pp. 317–323.

17. B. P. Gerkey, R. T. Vaughan, K. Støy, A. Howard, G. S.Sukhatme and M. J. Mataric, “Most Valuable Player: A RobotDevice Server for Distributed Control,” Proceedings of theIEEE/RSJ International Conference on Intelligent Robots andSystems (IROS 2001), Wailea, Hawaii (Oct. 29–Nov. 3, 2001)pp. 1226–1231.

18. Sun Microsystems Inc. Java IDL and RMI-IIOP Tools.Available: http://java.sun.com/j2se/1.5.0/docs/tooldocs/index.html#idl (2004).

19. R&D Center Yaskawa Corporation. Instructions for RTLabAPI (Ver 1.1.2). Yaskawa Robotics Technology R&D Dept(2004).