AI MATTERS, VOLUME 3, ISSUE 1 WINTER 2017

Evolutionary Online Learning in Multirobot SystemsFernando Silva (Instituto de Telecomunicacoes; fsilva@di.fc.ul.pt)Luıs Correia (BioISI; luis.correia@ciencias.ulisboa.pt)Anders Lyhne Christensen (Instituto de Telecomunicacoes; anders.christensen@iscte.pt)DOI: 10.1145/3054837.3054846

Introduction

Robots have the potential to replace mannedmachines and to carry out tasks in environ-ments that are either remote or hazardous,such as space, deep sea, or underground.However, to create intelligent, reliable, mo-bile robots, capable of operating effectivelyin a wide variety of environments, the lim-ited learning ability of robots needs to be ad-dressed (Brooks & Mataric, 1993). Mobilerobots are typically brittle: they are unableto adapt to changing environmental conditions(e.g., changes in terrain or lighting) or internalconditions (e.g., drift or permanent failure insensors and/or actuators), and to learn newtasks as they execute.

We study how learning can be achieved inrobotic systems through evolutionary algo-rithms (EAs), a nature-inspired approach thatmimics Darwinian evolution. Instead of man-ually programming the robots to carry out amission, an EA is executed onboard robotsduring task execution in order to synthesizeand continuously optimize the artificial brainor controller of each robot. This approach isknown as online evolution and can automat-ically generate the artificial intelligence thatcontrols each robot. However, despite thepotential for automatic robot learning, onlineevolution methods have not yet been able tosolve any but the simplest of tasks, and theapproach typically requires a prohibitively longtime to evolve controllers on real robots (sev-eral hours or days) (Silva, Duarte, Correia,Oliveira, & Christensen, 2016).

Contributions of Research

We have introduced a novel online EA calledodNEAT (Silva, Urbano, Correia, & Chris-tensen, 2015) for decentralized online evo-lution of artificial neural network (ANN) con-trollers in groups of robots that evolve in paral-

Copyright c© 2017 by the author(s).

lel and exchange candidate controllers to thetask. Contrarily to previous approaches, inwhich the controller structure is defined by thehuman experimenter, odNEAT evolves bothtopology and weighting parameters of ANNs.The algorithm starts with minimal networksand effectively complexifies them by addingnew neurons and new connections throughmutation. In this way, odNEAT can automat-ically find an appropriate degree of complexityto the current task (Silva, Urbano, et al., 2015).

We have extensively assessed the perfor-mance of odNEAT in a number of simulation-based studies, in which the algorithm wasshown to enable: (i) scalability (Silva, Cor-reia, & Christensen, 2015), as groups of dif-ferent size can leverage their multiplicity toachieve superior task performance and speedup evolution, (ii) robustness, as the controllersevolved can often adapt to changes in en-vironmental conditions without further evolu-tion (Silva, Urbano, et al., 2015), and (iii) faulttolerance, as robots executing odNEAT areable to adapt their behavior and to learnnew behaviors in the presence of sensorfaults (Silva, Urbano, et al., 2015). We haveadditionally developed different approaches tospeed up online evolution, including a tech-nique in which sub-behaviors can be prespec-ified in the neural architecture (Silva, Cor-reia, & Christensen, 2014), and racing tech-niques to cut short the evaluation of poor con-trollers (Silva, Correia, & Christensen, 2016).

For the real-robot experiments, we have im-plemented odNEAT in groups of Thymio IIrobots, each of which extended with a Rasp-berry Pi 2 single-board computer (see Fig. 1).The robots form an ad-hoc IEEE 802.11gwireless network, and communicate with oneanother by broadcasting UDP datagrams.We have successfully evolved controllers forcanonical tasks, including navigation and ob-stacle avoidance, homing, and aggregation.The controllers were evolved completely on-

23

AI MATTERS, VOLUME 3, ISSUE 1 WINTER 2017

Figure 1: Three of our Thymio II robots (two in thefront, one in the back), each of which extendedwith a Raspberry Pi 2 single-board computer.

line, in less than one hour, thereby showingthe potential of our approach and a path to-wards online evolution in a timely manner. Asthe final topic of the doctoral research, weare experimenting with evolutionary multi-levelcomposition of evolved control. Our final goalis to enable scalable and efficient synthesis ofsolutions for complex, multi-competence tasksbeyond the state of the art in the field, and tohelp realize the full potential of online evolu-tion.

Acknowledgments. This research waspartly supported by FCT under grantsSFRH/BD/89573/2012, UID/EEA/50008/2013,and UID/Multi/04046/2013.

References

Brooks, R. A., & Mataric, M. J. (1993).Real robots, real learning problems. InJ. H. Connell & S. Mahadevan (Eds.),Robot Learning (pp. 193–213). SpringerUS, Boston, MA.

Silva, F., Correia, L., & Christensen, A. L.(2014). Speeding up online evolution ofrobotic controllers with macro-neurons.In 17th European Conference on the Ap-plications of Evolutionary Computation(pp. 765–776). Springer, Germany.

Silva, F., Correia, L., & Christensen, A. L.(2015). A case study on the scalabilityof online evolution of robotic controllers.In 17th Portuguese Conference on Artifi-cial Intelligence (pp. 189–200). Springer,Germany.

Silva, F., Correia, L., & Christensen, A. L.(2016). Leveraging online racing andpopulation cloning in evolutionary multi-robot systems. In 19th European Confer-ence on the Applications of Evolutionary

Computation (pp. 165–180). Springer In-ternational Publishing, Switzerland.

Silva, F., Duarte, M., Correia, L., Oliveira,S. M., & Christensen, A. L. (2016). Openissues in evolutionary robotics. Evolu-tionary Computation, 24(2), 205–236.

Silva, F., Urbano, P., Correia, L., & Chris-tensen, A. L. (2015). odNEAT: An algo-rithm for decentralised online evolution ofrobotic controllers. Evolutionary Compu-tation, 23(3), 421–449.

Fernando Silva is adoctoral student and re-searcher at Instituto deTelecomunicacoes, Por-tugal. He studies how toenable the synthesis ofadaptive and robust robotsby taking inspiration frombiological principles atmultiple levels, and haspublished over 25 papers on

evolutionary computation, machine learning, androbotics.

Luıs M. P. Correia is as-sociate professor with habil-itation at Department of In-formatics of Faculty of Sci-ences of University of Lis-bon, Portugal. He lead theLaboratory of Agent Mod-elling (LabMAg), 2004-2014,and leads the MAS groupof BioISI from 2015. His

research interests are artificial life, autonomousrobots, and self-organised systems.

Anders Lyhne Chris-tensen is a tenured assis-tant professor at UniversityInstitute of Lisbon, Portu-gal, where he founded theBio-inspired Computationand Intelligent MachinesLab. Dr. Christensen haspublished more than 100papers on artificial intelli-

gence and robots, and he is on the editorial boardof three international journals.

24