Team Agent Behavior Architecture in Robot Soccer

Myriam Arias Ruiz and Jorge Ramirez Uresti

Abstract— One of the most challenging goals in artificialintelligence (AI) is to develop a team of autonomous agents withhuman-performance. In a team-based multi-agent system withcooperative and opponent team agents, the ability to build acoordinated system of game strategies can be useful to providethe agents with team plans to achieve common goals. Theseteam plans are used to make agents decide which behavior isthe best in a given situation, specially in a realistic setting ofsituated agents with local perception and uncertain domains.In this paper we present a Team Agent Behavior Architecture(TABA), an approach to coordinate the behavior of a multi-agent team. We introduce an autonomous and dynamic leaderagent capable of analyzing the current environment, includingthe opponent team, and select a team plan according to the stateof the world; this agent is selected among one of our cooperativeteam agents. The roles of the robots are not fixed and they candynamically exchange their roles with each other. The planselected is pre-defined and is communicated to the rest of theagents who execute it in a distributed fashion. We also proposea Soccer Strategy Description Symbols (SSDS) that is very simpleand intuitive. TABA is built using a hierarchical architecture.The first stage is responsible for leader agent selection; thesecond stage is for strategy selection; the third stage is for roleassignment and last stage is for tactic execution, in which robotsexecute their behavior commands based on their roles. Wehave successfully followed a Case-Based Reasoning approachfor strategy selection in the robot soccer domain, a simulationof the RoboCup four-legged league soccer competition. Initialresults of TABA are also discussed.

I. INTRODUCTION

In distributed artificial intelligence (DAI) processing takesplace not in a single algorithm but is distributed across anumber of agents, who receive information from its environ-ment, processes it, perform actions on that environment, andcollaborates with other agents to carry out a task or many.Each agent is autonomous and has a degree of expertise aboutsome area, knowledge of information or the behavior of thewhole system or part of this, and then, shares it with otheragents.

A robot is an example of an agent that perceives itsphysical environment through sensors and acts through ef-fectors. Robotics research is used in artificial intelligenceas a framework for exploring key problems and techniquesthrough a well-defined application like multi-agent systemsapplied to the robot soccer environment.

There are currently two major organizations promoting

M. Arias is a Masters degree student in Computer Science at Tecno-logico de Monterrey Campus Estado de Mexico, 52926 Atizapan, MexicoA01160325@itesm.mx

J. Ramirez is a Research-Professor, Department of Computer Scienceat Tecnologico de Monterrey Campus Estado de Mexico, 52926 Atizapan,Mexico juresti@itesm.mx

robot soccer as a robotic benchmark [11], [13]: FIRA1 andRoboCup2. The soccer game was selected for competitions,because, as opposed to chess, multiple players of one teammust cooperate in a dynamic environment and the implemen-tation of basic soccer skills such as approaching, controlling,dribbling and passing the ball, are a real challenge for AI [3],[5].

FIRA used previously the name MiroSot (Micro-RobotSoccer Tournament) that has now become a league withinFIRA, one of its main goals is to take the spirit of scienceand technology to the young generation and laymen to alsopromote the development of autonomous multi-agent roboticsystems. RoboCup is an international research initiative thatuses the game of soccer as a testbed to contribute to the state-of-the-art in artificial intelligence and robotics. The ultimategoal of the RoboCup project is by 2050 develop a team offully autonomous humanoid robots that can win against thehuman world soccer champion team. In the rest of this paperwe are going to focus on the RoboCupSoccer four-legged

league approach using the Sony AIBO ERS-7M23 robot. Atthe moment, this league is going through a transition fromthe four-legged Sony AIBO to Standard Platform League4,using the humanoid Aldebaran Nao5.

In multi-agent adversarial and cooperative settings, teamplans are particularly useful if they include some identifica-tion of potential patterns or oportunities to address a commongoal. Team plans with a combination of role assignment canaffect both accuracy and quality of the decision, the successand the speed of task execution for the agents operatingindependently or like a unit in an efficient way. Collaborationenables a team of agents to work together to address manydifferent issues of greater complexity than those addressedby agents operating individualistic [9].

This paper is intended to implement a team of cooperativeagents playing robot soccer. In section 2 we present anoverview of related research that has dealt with the topicof team coordination in a multi-robot system. Afterwards,in section 3, we describe our approach for application ofgame strategy architecture in a multi-agent environment andwe then, in section 4, describe the XML Behaviour Control

Simulator our test domain on which we tested our ideas.Finally, in section 5, we summarize and analyze additional

1Federation of International Robot Soccer Association,http://www.fira.net

2The Robot World Cup Soccer Games and Conferences,http://www.robocup.org

3Artificial Intelligence Robot, http://www.sonyaibo.net/4Standard Platform League, http://www.tzi.de/spl5Aldebaran Nao, http://www.aldebaran-robotics.com/

IEEE Latin American Robotic Symposium

978-0-7695-3536-4/08 $25.00 © 2008 IEEE

DOI 10.1109/LARS.2008.35

20

points for further research.

II. BACKGROUND

Robot soccer, as a multi-robot system, is a complexdomain involving multiple agents, a dynamic and uncertainenvironment, where agents need to collaborate against anopponent team in order to achieve a common goal, to scoregoals. In the work of [2] there were identified seven maintopic areas of a multi-robot systems: biological inspirations;communication; architectures, task allocation, and control;localization, mapping, and exploration; object transport andmanipulation; motion coordination; reconfigurable robots.We focus our work on architectures for multi-robot coop-eration specifically in decision-making.

A. Decision-making

One of the most important human skills is decision-making (judgment and choice) [1]. Both at a personal level,and in the context of organizations, decision-making skillsstrongly affects the quality of life and success. Most theoriesaccept the idea that decision-making consists of a number ofsteps or stages such as: recognition, formulation, generationof alternatives, information search, selection, and action.Furthermore, it is recognized that routine cognitive processsuch as memory, reasoning, and concept formation play aprimary role in decision-making.

B. Case-base reasoning

Case-based reasoning (CBR) is a model of reasoning thatincorporates problem solving, understanding, and learning,and integrates all of them with memory process. Thesetasks are performed using typical situations, called cases,already experienced by a system. A case may be definedas a contextualized piece of knowledge representing anexperience that teaches a lesson fundamental to achievingthe goals of the system. The four prime components of CBRsystem are retrieve, reuse, revise, and retain. These involvesuch basic tasks as clustering and classification of cases,case selection an generation, case indexing and learning,measuring case similarity, reasoning, and rule adaptation andmining [16]. Case retrieval, a measuring case similarity,is the process of finding within the case base the cases(s)that are closest to the current case. The case retrieved ischosen when the weighted sum of its features that matchthe current case is greater than other cases in the case base[15]. Some of the advantages of using this methodology are:reasoning in a domain with a small body of knowledge;reasoning with incomplete or imprecise data and concepts;providing flexibility in knowledge modeling; extending tomany different purposes; and specially reflecting humanreasoning. In our approach we use measuring case similarity

in the principal stage of strategy selection in order to rank thepotential strategy according to the state of the world againsta pre-defined list of strategies.

C. Game strategies

For human football teams, providing an adaptive playingstrategy, is relative easy, several criteria are taken into ac-count like position on the field of: our team, opponent team,ball, chances to attack or defend. For robot soccer severalmethodologies have been proposed for taking decisions. In[9] the aspects that a player needs to know are defined, likewhere to position itself, when to approach a ball, how toavoid an adversarial, when to act as attacker or a defender. In[4] it is proposed a implicit method of co-ordinating a teamof agents. This involves a list of multi-agent plans calledplays that a soccer expert can easily define for the system.They hypothesise that this approach will keep agents co-ordinated a reasonable percentage of the time even with theirdifferent perception. Also in [6], it is introduced the conceptof plays as a team plan, which combines both reactiveprinciples, that are the focus of traditional approaches forcoordinating actions, and deliberative principles. In additionit is presented the concept of a playbook as a method forcombining multiple team plans. The playbook provides aset of alternative team behaviors. Role assignment is aninteresting research problem in a complex multi-agent envi-ronment, the problem requires agents to decide the roles theyshould take based on real-time feedback from a dynamicallychanging environment. In [18], it is provided a new strategybased on the minority game, for assisting a team to performeffective role assignment. There are also many methodsbeing proposed about this area, such as genetic algorithms,neural networks, fuzzy logic [14], reinforcement learning[12], probabilistic models and strategies based on humanexperience.

III. TEAM AGENT BEHAVIOR ARCHITECTURE

On the four-legged league soccer environment, each robotbehaves autonomously using only available local informa-tion. This information is the sequence of every perception,agent’s sensor input, at any given instant. But it is usually in-feasible to implement a look up table that contains an actionfor every possible percepts sequence or state of the world

specially in fast-moving an uncertain real-time situationssuch as robot soccer. This is the main reason why we decidedto implement a set of pre-defined strategies or team plans,for the agents to follow, we call these team plans our strategy

base similar to playbook from other researchers, because thisconcept is used in the human soccer environment as well.

In addition there is no global controller like, in otherleagues such as in small-size. It could be viewed as a dis-tributed multi-robot system without a leader. It is necessaryto apply a role-based team structure in order to collaborateas a team, like in human behavior. Thus, a leader agent isproposed that has the ability to select a team plan to achievethe team’s goal.

A. The soccer field

We use a four-legged league soccer environment as atestbed for TABA, the soccer field was divided in thirty fiveregions in order to know where the agents and ball are on

21

the soccer field. Although their x, y coordinates give a veryspecific position, it is also useful to split the field up intoregions agents can make use of to define a overall situationand to implement strategies. The number of regions dependof the opportunity areas definition such as near of goals, nearof corners, center areas, etc,. Fig. 1 shows the soccer fielddivided.

Fig. 1. Soccer Field

B. Strategy Case Definition

A strategy is a long term plan of action designed to achievea particular goal, is a combination of moves for players in agame and can be decided by the current state of the game;strategy is differentiated from tactics or immediate actions; astrategy dealing with long periods of time (i.e. attack downthe right), a tactic deals with short periods of time, the actionon the game field itself (i.e. approach the ball) [7], [8], [10].In addition the formation and role assignment describe howplayers in a team are positioned on the soccer field andthe behaviors of each player; different formations and role

assignment can be used depending on the attitude of theteam (attacking or defensive style) and evolution of the game(winning or losing). A leader is necessary in order to makea decision about the strategy to perform given a situation inreal-time.

A strategy case consists of the description of the environ-ment in a past situation from a multi-agent’s point of viewand the actions each robot should perform for that situation.A strategy case has two main parts: (i) strategy definition and

conditions the general information of the strategy such as:name, id and time and the conditions for applicability suchas: team attitude and team and ball position; (ii) strategy

execution and termination list of behaviors for each robot(attacker, defender, supporter) to perform in a sequence andtermination conditions (goal scored, ball went out, timeout).Thus, a strategy case definition is given by:

C = {(a0 : V0), (a1 : V1), ..., (an : Vn)}, (1)

where Ci = (ai : Vi) is a pair of an attribute and its valuei.e C3 = {(teamattitude : kickoff), (ballposition :18), (player2position : 25), (player3position :22), (player4position : 18)}

A strategy base is a set of strategy cases, a set of all theproblems and all the solutions:

C∗t = {Cj‖Cj = {pj , sj}, pj ∈ P ∗

t , sj ∈ S∗t } (2)

where p is a problem and s the solution from a specificscenario t.

Table I shows an example of a strategy definition withkickoff attitude, also named as type; there are three maintypes of strategy: KickOff (0), Offensive (1) and Defensive(2). Attribute name is just for information purposes, time isset in seconds, the initial positions or conditions are specifiedin brackets.

TABLE ISTRATEGY DEFINITIONS AND CONDITIONS

STRATEGY CASE

Definition

ID K03TYPE 0NAME Right KickOffTIME 90

Conditions

PLAYER 1 [33]PLAYER 2 [25]PLAYER 3 [22]PLAYER 4 [18]BALL [18]

Table II shows an example of a strategy execution, positionand target for each player, also the termination conditionsare specified. If the strategy is selected, each player willperform the specified behavior i.e the agent in position[18] first approachBall and then sendPassToPosition [22],goToPosition [7] and waitPass. Finally, his final target isshootToGoal. All the players behaviors are executed in asequential way and the termination conditions are checkedall the time. Fig. 2 shows a graphical view of this strategyexecution, we propose a Soccer Strategy Description Symbols

(SSDS) as a notation useful to describe, test and modifyeasily strategy definition and execution. The symbols for thisnotation are showed in Fig. 3 and represent the state of theworld: teammates, opponents and ball, position and track arerepresented.

Fig. 2. Strategy definition and execution

C. Team Agent Behavior Architecture (TABA)

The Team Agent Behavior Architecture (TABA) that ischaracterized by hierarchical task decompositions, such as

22

TABLE IISTRATEGY EXECUTION AND TERMINATION

STRATEGY CASEID K03

Roles

PLAYER 1 defendGoal [33]PLAYER 2 goToPosition [8]

approachBall [8]passViability [7]shootToGoal [3]

PLAYER 3 waitPass [22]approachBall [22]sendPassToPosition [7]defendPosition [23]

PLAYER 4 approachBall [18]sendPassToPosition [22]goToPosition [7]waitPass [7]approachBall [7]passViability [8]shootToGoal [3]

Termination

checkTimeOutcheckGoalScoredcheckBallWentOutcheckAbortionMessage

Fig. 3. Soccer Strategy Description Symbols (SSDS)

in human soccer logistic planning and execution, applyingmost of the decision-making steps such as recognition,formulation, information search, selection, and action. Thearchitecture has four principal stages: (i) leader agent selec-tion, (ii) strategy selection, (iii) role assignment, and (iv)

tactic execution. In general, the process begins when thegame is started, every stage is related with the previousor next stage or the environment. The environment is allthe information affecting the game progress, such as teampositions, opponent positions, goal scored, etc., our approachget information from the environment and then it is modifiedby our architecture. Fig. 4 shows the Team Agent BehaviorArchitecture (TABA).

1) Leader Agent Selection: A leader agent is the agentwith the ability to make a decision about the best strategygiven a situation. The leadership is a dynamic role in our ap-proach and it is given to the agent who has a passive behaviorat that time, it is according to the ball position; i.e if

the ball is near of own goal, Attacker agent will be theagent leader. Algorithm 1 shows the LeaderAgentSelection

in general.2) Strategy Selection: When leader agent is defined,

he immediately requests information about of teammate’sperception, and builds a new situation to compare with

Fig. 4. Team Agent Behavior Architecture - TABA.

input : Game stateoutput: Leader agent

if gameState = kickOff OR ballPosition =1

opponentSide thenleaderAgent =GoalKeeper;2

end3

else4

if ballPosition = ownSide then foreach5

teammate i of the line N doFindDistanceBall(d)6

end7

leaderAgent =TeammateMaxDistanceToBall;8

end9

Algorithm 1: Leader Agent Selection

the strategy base, then analyze all the potential strategiesbased on similarity calculus, if it is found a similarsituation or strategy, this is then communicated to histeammate. If not, the agents perform a reactive behavioraccording to their primary role. We are implementing adecision-making algorithm to assign the heuristic and prob-abilistic based values to the different possible team plansaccording to the state of the world. The team plan or strategywith the maximum score will be selected. We are applyinga technique for measuring similarity among cases, a wellknown retrieval phase of CBR system, in order to comparethe potential strategy according to the state of the worldagainst our pre-defined team plans, and rank it accordingto their similarity. The equation

similarity(p, q) =n∑

j=1

f(pi, qi) × wi (3)

23

represent a situation for which p and q are two casescompared for similarity, p is the potential strategy to becompared with q the qualified strategy for specific state ofthe world; n is the number of attributes in each case, suchas teams position, opponents position, ball position; i is anindividual attribute from 1 to n; and wi is the feature weightof attribute i. The similarity calculation continues until allcases in the case library or playbook have been compared,and ranked. Similarities are usually normalized to fall withthe range 0 to 1, where 1 means a perfect match and 0indicates a total mismatch, but in our approach, similaritiesare normalized with the proportional range 0 to 100%, thusa strategy with a similarity of 75% or higher from one ofthe pre-defined team plan, will be selected. If the similarityis too low ( lower than 75%) no strategy will be selectedand the agents will execute a reactive behavior. This state, ofreactive behaviors, is finished when the agent leader requestsinformation to his teammates to begin a selection process, orwhen a goal is scored or the game is finished. Algorithm 2shows the StrategySelection in general.

input : New Situationoutput: Strategy to Play

setNewSituation;1

while reach the end of the strategy set do2

calculateDistanceFromNewSituation3

ToStrategyBase;mNearestNeighbors=minDistance;4

end5

foreach mNearestNeighbors i of the line N do6

calculateArithmeticMean7

end8

strategySelected =MAX(arithmeticMean);9

checkThresold(strategySelected);10

Algorithm 2: Strategy Selection

Figure 5 shows the SearchforSimilarity function. a) is theactual state of the world according to agent’s perception to besearched in the strategy base, b) is the formation with highersimilarity, so a strategy was found and the role assignmentwill be communicated.

3) Role Assignment: The collaboration between playersis achieved through the introduction of a formation for theteam, this formation decomposes the task space defining aset of roles. Role assignment is necessary to avoid collisionsof player going for the ball or a static player withoutassignment.

4) Tactic Execution: Tactics encapsulates a single robotbehavior determined by a specific strategy, is a set of actionsexecuted in a sequence and is created with the parametersdefined for the strategy for each robot. Most of the tacticsare applicable to a wider range of states of the world and areparametrized allowing reusable and redesigned performance.These behaviors will simply be executed until a terminationcondition applies.

Fig. 5. SearchForSimilarity function. a) actual state b) selected strategyaccording to actual state with the higher similarity

IV. XML BEHAVIOUR CONTROL SIMULATOR

In [17] we developed a platform for defining robot be-haviors using state machines modeled with XML. With thisany user can define a complex task even without knowingbasic programming details. The behaviors modeled withXML optimize programming the strategies directly in thenative language of the robot. We have great control overthe execution when simulating the agents because the usersees the behaviors exactly as they were described in theXML document. The different strategies and players couldbe easily incorporated in the environment definition throughXML, this accelerates the development of new strategies aswell as actions and events perceived by the robot, since itis only needed to update the listings of actions and eventsavailable. Also, a contract net approach was implementedand it was useful to analyse different cooperative behaviorsbetween agents and helps us to partition the problem inmany sub-stages keeping the general idea of strategy lessconfusing. Besides, a register and test of the behaviors allowus to define common potential strategies, it is the base of ourstrategy base. We are testing our architecture using the XML

Behaviour Control Simulator and the strategy definitions,conditions, role assignment and terminations are written inXML document, also configuration of the similarity function.Fig. 6 shows an image of the simulator with a footballenvironment.

24

Fig. 6. XML Behaviour Control Simulator . The simulator provides agraphic way to analyze a behavior or even navigate through the actionswith their associated behaviors or atomic actions.

V. CONCLUSION AND FUTURE WORK

A. Conclusion

In this paper, we described our cooperative strategy frame-work using RoboCup four-legged league as a test domain. Webriefly described the robot soccer environment as a roboticbenchmark also some of the research in decision-makingas well. We recognize that multi-robot teams in adversar-ial environments face several challenging goals. Our gamestrategy architecture consists of breaking down a probleminto several behavioral stages from low-level behaviors untilreaching high-level strategic behavior that handle with thefull domain complexity and achieve a common goal. We haveprovided a Team Agent Behavior Architecture for assistinga robot team to win in a robot soccer environment and couldbe implemented on any kind of teamwork environment. Wehave introduced a team strategy engine based on the conceptof leader agent, strategy selection, role assignment and tactic

execution and using a Soccer Strategy Description Symbols toshow a graphical view of the definition and execution of thestrategies. Multiple, distinct strategies or team plans can becollected into a strategy base and can be easily defined by aplay language where mechanisms for strategy selection withsome simple rules can enable the agent leader to improvethe team response to an opponent environment. Tests so farconfirm the effectiveness of the coordination between robotsto select a team plan through cases using a Case-BasedReasoning approach. Afterwards, our team jointly executethe actions of the selected strategy. For this approach, theplayers of a game need to have direct communication witheach other and then they make decisions according to aselection stage accomplished by a leader agent but theyare completely autonomous and is not necessary an externalcontroller. Although the communication is now limited onthe actual robot, we consider that the communication issuewill not be a big problem in the future.

B. Future Work

Future work will involve enhancing the strategy selectionalgorithm, applying mathematical analysis to study the be-havior of multi-robot system, select parameters that optimizeits performance, prevent instabilities and improvement thedesign process, testing with more agents and strategies, test-ing with real robots, testing in other domains like humanoids.

VI. ACKNOWLEDGMENTS

This work has been supported, in part, by the Secre-taria Relaciones Exteriores of Mexico grant ”Convocatoriade Becas para Extranjeros 2006” and by the Tecnologicode Monterrey Campus Estado de Mexico grant ”CatedraAgentes Virtuales y Roboticos en Ambientes de RealidadDual”.

REFERENCES

[1] Decision-making, 2007. http://tip.psychology.org/decision.html(17/10/2007).

[2] T. Arai, E. Pagello, and L. Parker. Guest editorial advances inmultirobot systems. In Robotics and Automation, IEEE Transactions,pages 655–661, 2002.

[3] M. Asada, H. Kitano, I. Noda, and M. Veloso. Robocup: Today andtomorrow what we have learned. In Artificial Intelligence, pages193–214, 1999.

[4] B. R. Baker, M. Reynolds, and W. Liu. Strategy specification forteamwork in robot soccer. In PCAR ’06: Proceedings of the 2006international symposium on Practical cognitive agents and robots,pages 129–140, New York, NY, USA, 2006. ACM Press.

[5] S. Behnke. Robot competitions - ideal benchmarks for robotics re-search. In IROS-2006 Workshop on Benchmarks in Robotics Research,2006.

[6] M. Bowling, B. Browning, A. Chang, and M. Veloso. Plays as teamplans for coordination and adaptation. In RoboCup 2003: Robot SoccerWorld Cup VII, pages 686–693. Springer Berlin / Heidelberg, 2004.

[7] C. Clapham and J. Nicholson. Strategy, 2005. http://0-www.oxfordreference.com (17/10/2007).

[8] L. D. Freedman, M. Matloff, and S. M. E. Howard. Strategy, 2007.(17/10/2007).

[9] V. Frias-Martinez, E. Sklar, and S. Parsons. Exploring auctionmechanisms for role assignment in teams of autonomous robots.In RoboCup 2004: Robot Soccer World Cup VIII, pages 532–539.Springer Berlin / Heidelberg, 2005.

[10] R. C. Giulianotti, J. D. Rollin, and B. Joy. Football, 2007.(17/10/2007).

[11] F. Groen, M. Spaan, and N. Vlassis. Robot soccer: Game or science.In Proceedings CNR-2002, 2002.

[12] L. P. Kaelbling, M. L. Littman, and A. P. Moore. Reinforcementlearning: A survey. Journal of Artificial Intelligence Research, 4:237–285, 1996.

[13] J. Malec. Implementing teams of agents playing simulated robotsoccer. citeseer.ist.psu.edu/malec00implementing.html (15/10/2007),2000.

[14] Q. Meng, X. Zhuang, C. Zhon, J. Xiong, Y. Wang, T. Wang, andB. Yin. Game strategy based on fuzzy logic for soccer robots. InSystems, Man, and Cybernetics, 2000 IEEE International Conferenceon, pages 3758–3763, 2000.

[15] S. K. Pal, T. S. Dillon, and D. S. Yeung, editors. Soft computing incase based reasoning. Springer-Verlag, London, UK, 2001.

[16] S. Shiu and S. K. Pal. Foundations of Soft Case-Based Reasoning.John Wiley & Sons, 2004.

[17] J. L. Vega, A. Junco, and J. Ramı́rez. Major behavior definition offootball agents through XML. In T. Arai, R. Pfeifer, T. Balch, andH. Yokoi, editors, Intelligent Autonomous Systems 9 (IAS-9), pages668–675, Tokyo, Japan, 2006. IOS Press. ISBN 1-58603-595-9.

[18] T. Wang and J. Liu. Evaluating the minority game strategy in agent roleassignments. In AAMAS ’05: Proceedings of the fourth internationaljoint conference on Autonomous agents and multiagent systems, pages1347–1348, New York, NY, USA, 2005. ACM Press.

25