Comparingtwoapproachesforgeneratinghomogeneousmulti‐agentteams

BachelorthesisArtificialIntelligenceRadboudUniversityNijmegen

EelkeBoezemans03519547thofJuly2008

CoordinatorsPimHaselagerIdaSprinkhuizen‐Kuyper

Contents

1.Introduction 3

2.Requirements 5

3.Task 6

4.Implementation 8

5.Experiment 9

6.Results 10

7.Conclusion 13

8.Literature 13

AppendixA 14

AppendixB 20

3

Abstract.Inthisbachelorthesiswestudiedtheeffectsofusingtwodifferentapproaches in generating homogeneous multi‐agent teams: the clonal andthea‐clonalapproach.Thea‐clonalapproachwashypothesizedtogeneratebetterperformingteamsontasksthatrequirethedevelopmentofrolesanddynamical role allocation. The task was implemented in a computersimulation. Results show that neither roles nor dynamical role allocationdeveloped. This made a comparison between the clonal and a‐clonalapproachimpossible,butdidshowwhyrolesanddynamicalroleallocationmightnothavedeveloped.

1.IntroductionMulti‐agentteamsaregenerallyeitherhomogeneousorheterogeneous.Bothtypeshavedifferentproperties,advantagesanddisadvantages,andareusedfordifferentpurposes.Theobjectofthisstudyisthecreationofhomogeneousmulti‐agentteams.AccordingtoMatariç(1997)homogeneousmulti‐agentsystemshavethreedefiningproperties.Inthesesystemsallagents:

1. areequal,e.g.havethesamehard‐andsoftware2. areautonomous.Thereisnocentralcontrolsystemcoordinatingthe

movementoftheagents3. sharethesameobjective.Theyworkasateamtofulfilacertaingoal.

Manyteamscanbegeneratedthathavethesethreeproperties.Anyteamshouldmaximizethethirdpropertywhilstnotviolatingthefirsttwo.Someteamsarebetterthanthisthanothers.Generatingahighperformingteamisacomputationalproblem.Duetothetotalnumberofpossibleteamsthatcanbegenerated,itisnotpossibletoconsiderallteamswithinreasonabletime.

1.1Randomselection

Asolutiontothisproblemwouldbetorandomlygenerateandtestteams.Thisapproachconsidersonlyafractionofallpossibleteams.Itislikelythatthebestperforminggeneratedteamisnotthebestifallpossibleteamswereconsidered.However,thisuncertaintyisnecessarytofindanappropriatesolutiontothecomputationalproblemwithinreasonabletime.

Intherandomlygenerateandtest‐approachteamsareselectedrandomlyandthusnouseismadeofthepropertiesofformergeneratedhighperformingteamstogeneratenewteams.Ifagoodperformingteamisfoundthechanceoffindinganothergoodperformingteamdoesnotincrease.

1.2Evolutionaryselection

Evolutionaryalgorithmsdotakeadvantageofformergeneratedteamstocreatenewteams.Anevolutionaryalgorithmbasicallyworksasfollows.Itstartswithtestingasetofrandomlygeneratedteams.Theworstperformingteamsaredeleted.Thebestperformingteamsareusedtogeneratenewteamsbymakinguseofevolutionarytechniquessuchascrossoverandmutation.Thenewteamsareevaluatedandthecycleisrepeateduntilacertainnumberofcyclesisreachedorthedesiredlevelofperformanceisfound.

4

Fromeverygenerationonlythebestteamscan“reproduce”andcreatenewteams.Everynewteamgeneratedbyevolutionhasahigherchanceofperformingbetterthanformergeneratedteams.

1.3Fromagentstoteams

Anyteamgeneratedisasolutiontothecomputationalproblemoffindingagoodperforminghomogeneousmulti‐agentteam.Anevolutionaryalgorithmgenerateshighperformingsolutionswithinreasonabletime.Itproducesthesesolutionsasblueprints,alsoknownasgenotypes.Thegenotypesareactuallynotteamspecifications,asonewouldexpect,butagentspecifications.Thisintroducestheneedfortwofunctionsthat(1)createteamsfromagentsspecificationsand(2)mapteamperformancetotheperformanceofasingleindividual.

Theseeminglymostsuitedwaytodothiswouldbetogenerateallmembersofateamfromasinglegenotype–theclonalapproach(seeFigure1,fromQuinn,2001).Becauseateamgeneratedbytheclonalapproachconsistssolelyoutofidenticalagents,teamperformanceequalsagentperformance.

Analternativetothisapproachisthe‘a‐clonal’approachinwhicheveryteammemberhasitsowngenotype(seeFigure2,fromQuinn,2001).Inthisapproachitismuchhardertorelateteamperformancetotheperformanceofasingleindividual.Becauseateamconsistsoutofdifferentgenotypesitisnotclearwhateachagent’scontributionistotheteamperformance.ArathercomputationallyheavysolutiontothisproblemiscalledFitnesssharing.Everygenotypeistestedinseveraldifferentteams,eachconsistingoutofdifferentgenotypes.Theperformanceofasinglegenotypeiscalculatedastheaverageperformanceoftheteamsithasparticipatedin.

Thereseemstobenoobviousreasontousethea‐clonalapproachtogeneratehomogeneousteams.Itevaluatesmostlyillegal(heterogeneous)teamsandiscomputationallymuchharderthantheclonalapproach.

1.4Rolesanddynamicalroleallocation

Surprisingly,Quinn(2001)showsinatworobotcoordinatedmovementtaskthatthebestteamgeneratedbythea‐clonalapproachoutperformsthebestindividualgeneratedbytheclonalapproach.Inthistasktwoidenticalagentsmustmovetogetherasfaraspossible.Theagentshavelimitedperception;they

5

easilylosetrackofeachother.Teamperformancewillthusbeverylowiftheagentsdonotcooperate.Itiscriticalthatatonepointoneagenttakestheleadandtheotheragentfollows.Thisrequirementforcooperationandroleallocationincreasesthedifficultyofthetaskgreatly.Becausetheteamishomogeneousitrequiresasingleagentto(1)performdifferentrolesintheteamandto(2)‘dynamically’switchbetweentheseroles.

Genotypesintheclonalapproachareevolvedonlyinteamstogetherwithcopiesofthemselves.Becauseonlyonegenotypeisusedforthecreationofeveryteammember,asinglegenotypemustrepresentalltherolesthatshouldbeperformedbytheagents.Theclonalapproachrequirestheevolutionaryalgorithmtodevelopmultiplerolesandamechanismtoswitchbetweentheserolesinasinglegenotype.

Theadvantageofthea‐clonalapproachovertheclonalapproachisthatitallowsforaseparatedevelopmentoftherolesandthemechanismtoswitchbetweentheroles.Thisseparationindevelopmentisduetothefactthatdifferentgenotypesevolveintodifferentroles.Ahighteamperformancecanbeachievedearlyintheevolutionaryprocessbecausethenecessaryrolesaregeneticallydetermined.Laterwhentheevolutionaryalgorithmconvergesintoasinglegenotypeamechanismevolvesthatallowsasinglegenotypetobeabletoperformmultipleroles.

1.5.Aimofthisstudy

Theaimofthisstudyistoinvestigatetheperformanceoftheclonalandthea‐clonalapproachingeneratinghomogeneousmulti‐agentteamsfortasksthatrequirethedevelopmentofrolesanddynamicalroleallocation.Weinvestigatethetwoapproachesbytestingthefollowinghypothesis,derivedfromQuinn(2001):

Thebestindividualgeneratedbythea‐clonalapproachoutperformsthebestindividualgeneratedbytheclonalapproachusedinahomogeneousmulti‐agentteamonataskthatrequiresdynamicalroleallocation.

Fromanexperimentalperspectivethetestingapproach(clonal/a‐clonal)istheindependentvariable.Thedependentvariableisthefitnessscoreoftheteamcreatedfromthebestperforminggeneratedindividual.

Inthefollowingsectionstherequirementsnecessarytotestthehypothesisareanalysedandatestingenvironmentisproposedthatmeetstheserequirements.

2.RequirementsDynamicalroleallocationistheessentialmechanismthatunderpinsthesuccessofthea‐clonalapproachovertheclonalapproach.Thea‐clonalapproachisexpectedtobemorecapableoffindingsolutionsfortasksthatrequiresuchamechanismtodevelop.

InacontrolstudyQuinn(2001)showsthatthea‐clonalapproachdoesnotoutperformtheclonalapproachifthetaskistootrivial.Itisthereforecriticalthattheproposedtestingenvironmentrequiresthedevelopmentofrolesand

6

dynamicalroleallocation.Asmuchasthetaskrequiresthedevelopmentofthesetwoaspectssomusttheagentcontrolstructurebeabletorepresentsuchbehaviour.

3.TaskTakingtheserequirementsintoaccountthefollowingtaskisproposedthatisconsideredbeingcapableoftestingthehypothesis.Thetaskdescribedisessentiallyapackageroutingtaskinwhichagentsmusttransportpackagesfromsourcestogoalsasfastaspossiblewithinapredefinedtimeframe.InFigure3animageofthetaskenvironmentisshown.

Figure3:Avisualrepresentationoftheenvironmentconsistingofagents(redandgreen),sources(pink),goals(blue)andpackages.Thegoalfortheagentsistotransportasmanypackagesfromsourcestogoalswithinapredefinedtimeframe.

3.1Specification

Agentsinhabitthetaskenvironmentandaretheonlyautonomousobjectsinthesensethatonlytheycanchangetheirstate.Agentsflyaroundandchangetheirflyingdirectionbasedontheirperceptionoftheenvironment.Whenanagentcollideswithanotheragentandoneofthetwotransportsapackageitisexchangedbetweenthem.

Atirregularintervalssourcesofferpackages.Agentsperceivesourcesthatofferpackagesanduponcollisionbetweentheagentandthesourcewithapackageitisexchangedbetweenthem.Whentheagentistransportingapackageanditcollideswithagoal,thepackageisdroppedoffatthegoal.Thenumberofpackagesthataresuccessfullytransportedbytheagentsfromsourcestogoalswithinapredefinedtimeintervaldenotesthefitnessoftheteam.Thebettertheteamisatworkingtogethertotransportpackagesthehigherteamperformanceandtheobtainedfitnessscore.

3.2Agentcontrolstructure

Theagentcontrolstructure,derivedfromSpector,Klein,PerryandFeinstein(2005),isbasicallyasinglelayerneuralnetwork.Theagentperceivestherelativelocationofotherobjectsintheenvironmentrepresentedasvectors.

7

Figure4:Everyagentperceivesobjectsintheenvironmentasvectors.ShownarevectorsforClosestAgentwithPackage(greenbox),ClosestGoal(top‐leftbluecircle),ClosestSourcewithPackage(purplebox)andClosestSourcewithoutPackage(purplecircle).

Figure4showstheexampleenvironmentofFigure3addedwithfourinputvectors:ClosestAgentwithPackage,ClosestGoal,ClosestSourcewithPackageandClosestSourcewithoutPackage.Thecontrolstructureweighsthesevectorsandsumstheweightedvectorstogetherwiththecurrentflyingdirection.Theagentcontinuouslyupdatesitsflyingdirectionbychangingitaccordingtotheinputitreceives.

InFigure3and4twosources,twogoals,twoagentsandtwopackagesareshown.Theseamountsarearbitraryandcanbeconfiguredtoanynumbertosuitthetaskandcomputationalresources.

3.3Cooperation

Cooperationbetweentheagentsisencouragesbyslowingdownanagentsignificantlywhentransportingapackage.Althoughthismakesitpossibleforeveryagenttodeliverapackagewithoutcooperation,ittakesmuchlongerthanwiththehelpfromanotheragent.Cooperationdecreasesthetimeneededtotransportasinglepackageandthusincreasesteamperformance.

3.4Roles

Thetaskisexpectedtorequirethedevelopmentofthesetworoles:

• Locatingapackage.Assoonasapackageisofferedatasourceoneoftheagentsfliestowardsthesourceandpicksitup.Becauseonlyoneoftheagentscanpickupthepackage,it’snotbeneficialforbothagentstoflytowardsthesamesourceassoonasitoffersapackage.

• Helpinganotheragenttransportingapackage.Whenanotheragentistransportingapackagetoagoal,it’sbeneficialforbothwhentheycooperateintransportingthepackagetoagoallocation.

Theserolesareexpectedtodevelopbecauseagentsareexpectedtocooperate.Cooperationisdefinedasanybehaviourthatleadstoafastertransportationofpackages.Fastertransportationisachievedwhentheagentstransportthepackageinturnandthussharingtheslowingburdenofthepackage.

8

4.ImplementationThetask,evolutionarystrategyandthetwoapproachesneedtobeimplementedinsuchawaythatitiseasytounderstandwhattheinfluenceofeachcomponentis.Thetaskalsorequiresacertainamountofflexibilitysothatitisrelativelyeasytochangethetaskconfiguration,orotherparametersthatcontrolthesimulation.

4.1Model

AModelisdesignedtoimplementthesimulation.TheModelhasthreemaincomponents(seeFigure5):

Model=<Evolution,Tester,Environment>

Figure5:AvisualrepresentationoftheModelwithitsthreemaincomponents:Evolution,TesterandEnvironment.

ThetwomaincomponentsoftheModelareEvolutionandEnvironment.TheTestercomponentsitsinbetweenandhandlesinputandoutputofthesecomponents.

Evolutionisanevolutionarymechanismthatevolvesgenotypes.Genotypesareparametersettingsforagentsintheenvironment.Afterproducingafreshgenotypeitremainsidleuntilafitnessscoreisgivenback.WhenthegenotypesfitnessisreceivedEvolutioniteratesthroughthewholeprocessagain.

Environmentisatwo‐dimensionalspacethatcontainsmovingagentsthatinteractwithotherobjectsinthespace.Italsodescribesafitnessfunctionthatdenoteshowgoodtheperformanceoftheagentsisaccordingtoapredefinedtask.TheEnvironmentissetupwithateamtotest.ItproducesafitnessvaluewhichispassedtotheTester.

TesterconnectsEnvironmentandEvolution.Itreceivesuntestedgenotypesfromevolution’sgenepool,generatesamulti‐agentteaminaccordancetotheclonalora‐clonalapproachandteststheteaminEnvironment.Afteranumberofteststhegenotype’sfitnessisestablishedandreportedbacktoEvolutiontofacilitatetheevolutionaryprogress.

Foramoredetaileddescriptionofthemodel,seeAppendixA.

Model

genotypefitness

teamfitness

genotype team

Evolution

Genepool,EvolutionaryStrategy

Tester

Generator,Configurator,Tester

Environment

Agents,World,Task,Fitnessfunction

9

4.2Software

Thesimutationisimplementedinbreve,anopen‐sourcemulti‐agentsimulationframework(Spector,Klein,Perry,Feinstein,2005).Brevecanbeusedtoimplementthegeneticalgorithmandthevisualsimulation.Thereisalsoacommandlineinterfaceavailablethatallowssimulationstoberunwithoutvisuals,vastlyincreasingtherunningspeed.

5.ExperimentThetaskisconfiguredasfollows:everysimulationcontainstwoagents,twosourcesandtwogoals.BecauseonlyonepackageisallowedtoexistintheEnvironmentatanytime,thereisamaximumofonlyonepackagetodeliveratanytime.Thisencouragescooperationbetweenagents.ThelocationsofallobjectsarevariedinthreeconfigurationsthathavethesetupaspicturedinFigure3.Theconfigurationsdifferinthedistancebetweenthestartingpositionoftheagentsandthepositionofthesourcesandgoals,andthusareincreasinglydifficult.Everygenotypeistestedineveryconfiguration.

Theevolutionarystrategythatisusedistheµ+1strategy.Thegenepoolisfilledwith30randomgenotypes.Afterallhavebeentestedanewgenotypeisgeneratedfromthegenepoolusingrankbasedselection.Thenewgenotypereplacescurrentworstgenotypeinthegenepool.After500generationsthebestgenotypeisconsideredasthesolutionoftherun.Allsimulationsareruneitherinclonalora‐clonalmode.Every10generationsthecurrentbestgenotypeofbothapproachesisevaluatedinahomogeneousteamtocomparebetweenbothapproaches.

Duetolackofcomputationalresourcesonlyfiverunsareexecutedforeachapproach.AllparametersettingsareshowninTable1.SeeAppendixBformoreparametersettings.

Numberofrunsexecutedperapproach 5

Numberofconfigurationsonwhicheachgenotypeistested 3

Numberofgenerationsperrun 500

Numberofnewgenotypesgeneratedeverygeneration 1

Time(simulationiterations)usedtotestgenotypeonconfiguration 1000

Numberofgenotypesingenepool 30

Numberoftrialsafterwhichbestgenotypeisevaluatedhomogeneously 10

Numberofsources 2

Numberofgoals 2

Numberofagentsinateam 2

Numberofgenesingenotype/Numberofvectorweights 7

10

Minimalagentvelocity 0.1

Maximalagentvelocity 1

Maximumnumberofpackagesallowedintransport 1

Speedslowingfactorforcarryingapackage(persimulationiteration) ‐0.01

Speedincreasingfactorfornotcarryingapackage(persimulationiteration) +0.01

Table1:Parametersettingsofallsimulations

6.ResultsInFigure6theobtainedfitnessvaluesofthebestgenotypesafterfiverunsarepicturedforbothapproaches.Itisclearlyvisiblethatthegenotypesproducedbythea‐clonalapproachhaveamorevaryingperformancethanthegenotypesproducedbytheclonalapproach.Theall‐timebestperforminggenotypesforbothapproachesgainafitnessvalueof6.

Figure6:Fitnessvaluesofbestperforminggenotypesfrombothapproaches.Thea‐clonalapproachproducesmorevaryingresultsthantheclonalapproach,butbothproduceagenotypewithafitnessvalueof6.

Theresultsshowthatthea‐clonalapproachdidnotoutperformtheclonalapproach.Bothapproachesgeneratedteamswithafitnessvalueof6.Forthisreasonnostatisticalanalysiswasperformedonthedata.Table2summarizestheperformanceofbothapproachesandgivesanindicationofthedifferenceincomputationalloadbetweenthetwoapproaches.

Clonalapproach A‐clonalapproach

Bestfitness 6 6

Averagefitness 5.39 5.04

Standarddeviation 0.37 1.15

Averageapproximateruntime 10.5Hours 21Hours

Table2:Resultsofbothapproachesafter5runsperapproach

6.1Observedbehaviour

Thesimulationsshowthattheagentsdonotadapttomultipleroles,butinsteadadapttojustonesinglerole,carriedoutbybothagents.Thisrolecanbedescribedasfollows:

0

2

4

6

8

1 2 3 4 5

Fitness values of best genotypes after each of five runs

Clonal

A-clonal

11

Stayclosetotheotheragentandthenearestgoal.Whenapackageisofferedatthesource,gotoit.Whenthepackageispickedupreturntothegoallocation.

Thisroleisamixbetweenthetworolesthatwereexpectedtodevelop.Itincorporatesbothlocatingapackageatasourceandhelpinganotheragenttransportapackage.Theweightdistributionintheagentcontrolstructureofthebestgenotypesfrombothapproachesshowshowthissingleroleisrepresented.Figure7showsatypicalevolutionofthebestgenotypewithinasinglerunforboththeclonalanda‐clonalapproach.

A‐clonalapproach(run1):Bestgenotypefitnessvaluesandweights

Clonalapproach(run2):Bestgenotypefitnessvaluesandweights

Figure7:Theevolutionofthebestgenotypeinbothapproaches.Theweightsaretypicalandinlinewiththeoptimalrolebehaviourtheagentsshow.

InbothapproachesthevectorlocatingtheSourcewithPackagevectorhasthehighestweightandtheGoalvectoralwayshasalowerweightthanSourcewithPackage.Thisisinlinewithobservedsinglerolebehaviour.IfnopackageisofferedatanysourcethanSourcewithPackageis0andtheagentscirclethegoal.Whenapackageisoffered,thesourceofferingthepackagehasastrongerattractionandtheagentsbothmovetothesource.Assoonasoneoftheagents

-2

-1

0

1

2

3

4

5

1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96

A-clonal 1: best genotype fitness & weights

Fitness

Agent with Package

Agent without Package

Source with Package

Source without Package

Goal

Random

World Center

-2

-1

0

1

2

3

4

5

6

7

1 6 11 16 21 26 31 36 41 46

A-clonal 1: best genotype fitness & weights

Fitness

Agent with Package

Agent without Package

Source with Package

Source without Package

Goal

Random

World Center

-1

-0,5

0

0,5

1

1,5

2

2,5

3

3,5

1 6 11 16 21 26 31 36 41 46

A-clonal 1: best genotype fitness & weights

Fitness

Agent with Package

Agent without Package

Source with Package

Source without Package

Goal

Random

World Center

-1

0

1

2

3

4

5

6

7

1 7 13 19 25 31 37 43 49

Clonal 2: best genotype fitness & weights

Fitness

Agent with Package

Agent without Package

Source with Package

Source without Package

Goal

Random

World Center

-1

0

1

2

3

4

5

6

1 7 13 19 25 31 37 43 49

Clonal 2: best genotype fitness & weights

Fitness

Agent with Package

Agent without Package

Source with Package

Source without Package

Goal

Random

World Center

-1

0

1

2

3

4

5

6

7

1 7 13 19 25 31 37 43 49

Clonal 2: best genotype fitness & weights

Fitness

Agent with Package

Agent without Package

Source with Package

Source without Package

Goal

Random

World Center

-1

0

1

2

3

4

5

6

1 7 13 19 25 31 37 43 49

Clonal 2: best genotype fitness & weights

Fitness

Agent with Package

Agent without Package

Source with Package

Source without Package

Goal

Random

World Center

-1

0

1

2

3

4

5

6

7

1 7 13 19 25 31 37 43 49

Clonal 2: best genotype fitness & weights

Fitness

Agent with Package

Agent without Package

Source with Package

Source without Package

Goal

Random

World Center

-1

0

1

2

3

4

5

6

1 7 13 19 25 31 37 43 49

Clonal 2: best genotype fitness & weights

Fitness

Agent with Package

Agent without Package

Source with Package

Source without Package

Goal

Random

World Center

12

picksupthepackageSourcewithPackagebecomes0again.NowGoalhasthehighestattractionandthustheagentsmovetothegoal.Theagentsremainthereuntilanotherpackageisofferedatasourceandtheprocessrepeats.TheweightdistributionsinFigure7explainshowthesinglerolebehaviourisrepresentedinagents.

Thereasonthisbehaviourdevelopsisbecauseitistheoptimalstrategywithinthemodelledenvironment.Developmentofmultiplerolesorevendynamicalroleallocationdoesnotincreaseteamperformanceonthetask.Therearevariousaspectsofthesimulationthatcouldhavecausedthis.

6.2Fitnessfunction

Theusedfitnessfunctionrewardsagentsforpositivebehaviour.Themorepackagestransportedthehigherthefitnessvalueoftheteam.Nonegativefeedback,i.e.thedistancetravelledbytheagents,wasincorporatedinthefitnessfunction.Thelackofnegativefeedbackallowstheagentstotravelthesamedistancetogetherwithoutpunishment.Ifnegativefeedbackwereimplementedinthesimulationtravellingtogetherwouldreceivenegativefeedbackfromthefitnessfunction.Inthatcasecooperativebehaviourwouldstillbefavoured,butonlyiftheagentswouldnottraveltogether.Itcanbeimaginedthatthiswouldaidthedevelopmentofmultipleroles.

6.3Agentcontrolstructure

Anotherreasonwhynomultiplerolesdevelopcouldbethattheagentcontrolstructureisnotcapableofrepresentingrolesorevendynamicalroleallocation.Thecontrolstructureusedinthissimulationcanbeconsideredasasinglelayerneuralnetwork.Itispossiblethatonelayerisnotcomplexenoughtorepresentsophisticatedbehaviourandcanonlyleadtosinglerolebehaviour.

Inthecurrentcontrolstructureinputvectorsareweighedandcompeteforinfluenceintheflyingdirectionoftheagent.Nointeractionbetweeninputsispossible,sonocombinedevaluationofmultipleinputsispossibleeither.Thiscausestheagenttobeblindforeventsorcombinationsofeventsintheenvironmentthatindicateacertainrelevantstateoftheenvironment.

Thecurrentcontrolstructureisnotcapableofrepresentingmanystatesoftheenvironment.Becausetheagentcontrolstructureisreactive,perceptionisdirectlyrelatedtoaction.Consequently,thebehaviouroftheagentisaproductofthestatesitcanrepresent.Iftheagentisabletorepresentmorestatesitispossiblethattheagentisalsocapableofshowingmorecomplexbehaviour.

6.4Scopeofperception

Theagentsinthissimulationhaveanunlimitedscopeoftheworld.Theirperceptionexistsoutofinputvectorswhichrepresentthelocationofotherobjectsintheworld,suchastheclosestotheragent,sourceorgoal.Anagentknowsthestateofalmosttheentireworldnomatterwheretheagentitselfislocatedwithinthisworld.Becauseofthis“globalperception”thereisalmostnouncertaintyintheperceptionoftheagent.Rolesarelikelytodevelopifagentsdependoneachother.Withaglobalscopeanagentdoesnotneedanotheragentforanything,buttoincreasethespeedoftransportation.Noinformationis

13

exchangedbetweenagentsinateam,becauseeveryagentknowstheentirestateoftheworld.Ifthescopeoftheagentswerelimitedagentswouldneedtocooperatetoexchangeinformation,e.g.thelocationofasourcewithapackage.Thiscouldcontributetothedevelopmentofroles.

7.ConclusionInthisbachelorthesiswestudiedtheeffectsofusingtwodifferentapproachesingeneratinghomogeneousmulti‐agentteams:the“standard”clonalapproachand“alternative”a‐clonalapproach.Thea‐clonalapproachwashypothesizedtogeneratebetterperformingteamsontasksthatrequirethedevelopmentofrolesandamechanismtoswitchbetweentheseroles,dynamicalroleallocation.Asimulationenvironmentandataskwereimplementedandexperimentswereruninordertotestthishypothesis.Theresultsshowthatneitherrolesnordynamicalroleallocationdeveloped.Thismakesacomparisonbetweentheclonalanda‐clonalapproachimpossiblebecausedevelopmentofsuchbehaviourisafundamentalelementofthehypothesis.

Althoughthisresearchhastriedtogaininsightintotheworkingsofthea‐clonalandclonalapproaches,itremainssomewhatunclearwhatexactlyconstitutesanenvironmentinwhichrolesanddynamicalroleallocationarerequiredtodevelop.Furtherresearchisnecessarytodiscoveranddefinethepropertiesofsuchanenvironment.Wehypothesizethatinourstudyrolesanddynamicalroleallocationdidnotdevelopduetoanumberoffactors,suchasthetask,thefitnessfunction,theagentcontrolstructureandthescopeofperceptionoftheagents.

Ofthesefactors,thefitnessfunctionandagentcontrolstructurearethetwomostobviousaspectsworthevaluatingfirst.Thefitnessfunctionshouldincorporatenegativefeedbackencouragingtheagentstocooperatebutnottodevelopthesamerole.Theagentcontrolstructureshouldbedesignedtobeabletoperceiveandactonmoresubtledifferencesbetweenenvironmentalstates.Combinationsofinputvectorsmustbeweighedandusedastriggersforcertainbehaviours.Thiscanonlybeachievedifthecontrolstructureismorecomplex.Addingmorelayerstothecurrentsinglelayernetworkisalikelyapproach.

Withtheseamendmentsitispossiblethatrolesanddynamicalroleallocationdodevelop.Thedevelopmentofthesepropertieswouldenabletheclonalanda‐clonalapproachtobecomparedingeneratinghomogeneousmulti‐agentteams.

8.LiteratureMateriç,M.(1997).Behaviour‐basedcontrol:Examplesfromnavigation,learning

andgroupbehaviour.JournalofExperimentalandTheoreticalArtificialIntelligence,9(2‐3),

Quinn,M.(2001).AComparisonofApproachestotheEvolutionofHomogeneousMulti‐RobotTeams.Proceedingsofthe2001CongressonEvolutionaryComputationCEC2001,IEEEPress,128‐135

Spector,L.,Klein,J.,Perry,C.,Feinstein,M.(2005).EmergenceofCollectiveBehaviorinEvolvingPopulationsofFlyingAgents.InGeneticProgrammingandEvolvableMachines,6(1),111‐125.

14

AppendixA:ModeldescriptionThissectiongivesanexactdescriptionoftheModel.TheModelcontainsthreeelements:Environment,EvolutionandTester.Eachoftheseelementsandtheirsub‐elementsaredescribedindetailtogiveanexactaccountoftheimplementation.

1.Environment

Environment=<World,Agents,Task>

TheEnvironmentcontainsthreemainobjects:aspatialvirtualWorld,AgentsthatmovearoundwithinandinteractwiththeWorldaTaskthatspecifiesthedesiredstateoftheWorld.

1.1World

- ThevirtualspaceoftheWorldistwo‐dimensional.

- TheWorldcontainsseveraldifferentobjects.Everyobjecthasastatethatisalteredbypre‐seteventsorinteractionwithotherobjects.

- Passiveobjects(Packages,SourcesandGoals)cannotexecuteactionsorinitiateinteractionwithotherobjectsandgetalteredbypre‐seteventsoractionsofactiveobjects.

- Activeobjects(Agents)functionautonomouslyandperformactionsonotherobjects.

- ThestateofeveryobjectisupdatedeveryiterationoftheEnvironment.

1.1.1Package

- APackageisanobjectthatcanbetransferredfromAgenttoAgent,SourcetoAgentandAgenttoSource.Itispassiveinthesensethatitcannotperformactionsthataltersitsownstate.

- ThenumberofPackagesthatexistsintheWorldcannotexceedNpackagesAllowedInWorld.

- PackagescanonlycomeintotheWorldatSources.NpackagesOffereddenotesthenumberofPackagesinstantiatedatSources.

- PackagesdisappearatGoalswhendeliveredbyanAgent.NpackagesDelivereddenotesthenumberofPackagesdeliveredatGoals.

1.1.2Source

- ASourceisapassiveobjectthatoffersPackages.

- ThereareNsourceSourcesintheWorld.TheTester’sConfiguratordeterminesthelocationofeverySource.

- NpackagesOffereddenotesthenumberofPackagesofferedbyallSourcesintheWorld.IfaSourceoffersaPackageNpackagesOfferedisincreasedbyone.

15

- ASourceoffersaPackageontwoconditions:

o thenumberofPackagescurrentlyexistingwithintheWorlddoesnotexceedNmaxPackagesAllowedInWorld.

o Arandomamountoftime(notbiggerthanTmaxWaitingTimeToOfferPackage)haspassed.

1.1.3Goal

- AGoalisapassiveobjectthatreceivesPackagesfromAgents.

- ThereareNgoalGoalsintheWorld.TheTester’sConfiguratordeterminesthelocationofeveryGoal.

- IfaGoalreceivesaPackageNpackagesReceivedisincreasedbyone.

1.2Agents

AnAgentisanobjectthatexistswithintheWorldbutisdifferentfromtheSource,GoalandPackageinthesensethatitactivelychangesthestateoftheWorld.Itisconsideredautonomousandactingtoobtainacertainobjective.

NAgentdenotesthenumberofAgentsthatexistwithintheEnvironment.

EveryAgenthasthefollowingproperties:

- AnAgenthasaspatiallocationandmovesaroundwithinthetwodimensionalEnvironment.BoththeAgent’slocationandflyingdirectionarerepresentedbyvectorsthatgetupdatedeveryiterationoftheEnvironment.

- ThereisnomaximumvalueforthelocationofanAgent.TheflyingvelocitymustbebetweenminVelocityAandmaxVelocityA.

- AnAgent’senergy(betweenminEnergyAandmaxEnergyA)affectstheflyingvelocityoftheAgent.

- AnAgentcantransportaPackage.TransportingthePackagelowerstheAgent’senergybyenergyStepAperiteration.Viceversa,nottransportingaPackageraisestheAgent’senergybyenergyStepA.

- APackageisexchangedbetweenanAgentandanotherobjectinthefollowingsituations:

o AnAgentnottransportingaPackagecollideswithaSourceofferingaPackage.

o AnAgenttransportingaPackagecollideswithaGoal.

o AnAgentcollideswithanotherAgentandoneofthemtransportsaPackage.

16

- AfteranAgentexchangesaPackagewithanotherobject,beinganotherAgent,SourceorGoal,itcannotexchangeforacertainnumberofiterationsdenotedbycollisionTransferDelay.

- AnAgentisawareofthelocationofthefollowingotherobjectsintheEnvironment.ThesearetheAgent’sinputvectors:

o ThenearestotherAgentwithaPackage

o ThenearestotherAgentwithoutaPackage

o ThenearestSourceofferingaPackage

o ThenearestSourcenotofferingaPackage

o ThenearestGoal

o Randomlocation

o CentreoftheWorld

- ThecontrolstructureofanAgentspecifiesaweightforeveryinputvector.EveryiterationtheAgent’sflyingdirectioniscomputedbyweighingtheinputvectorstogetherwiththeAgent’scurrentflyingdirection.

1.3Task

TheTaskspecificieswhattheAgentsintheWorldshoulddo,thuswhatbehaviourisgoodandwhatbehavourisnot.ThisvaguenotioniscapturedbytheFitnessfunctionthatmapsAgents’behaviourtoaperformancevalue.

Taskhasthefollowingproperties:

- TheFitnessfunctiondefinestheperformanceoftheAgentsasthenumberofPackagesdeliveredatGoalssuccessfully.ThehigherthenumberofPackagesdelivered,thehigherthefitnessoftheteamofAgents.

- TheTaskendswhenacertainamountoftimehaspasseddefinedbyTtimeToTestTeam.

2.Evolution

Evolution=<Genepool,EvolutionaryStrategy>

Evolutionhastwomaincomponents.GenepoolcanbeconsideredasadynamicalcomplexdatacontainerandtheEvolutionaryStrategyasthesetofactionsthatareperformedonthedataeveryevolutionaryiteration.

2.1Genepool

- AGenepoolisacollectionofGenotypes.ThenumberofGenotypesintheGenepoolisequaltoNgenotype.

17

- ThereisonlyoneGenepoolatanygiventime.ThestateoftheGenepoolchangeseveryevolutionaryiteration.InsuchaniterationthecurrentGenepoolisextendedwithnewGenotypesthatarecreatedfromGenotypeswithinthecurrentGenepool.

- AGenotypeconsistsofanumberofGenesdenotedbyNgenesPerGenotype.NgenesPerGenotypeisequaltoNagentWeights,thenumberofweightsinthecontrolstructureofanAgent.

- AGeneisafloatingpointnumberanditsvalueiswithintheintervalofminGeneValueandmaxGeneValue.

- ForeveryGenotypeaFitnessvaluecanbecalculatedthatindicatesthequalityofthegenotypewithrespecttotheTaskdefinedintheEnvironment.

2.2EvolutionaryStrategy

IftheGenepoolisempty,initializetheGenepoolwithrandomGenotypesandtestallGenotypestoobtainafitnessvalueforeachGenotype.

WhengivenallGenotypesaretestednewoffspringisgeneratedbyapplyingtheµ+1algorithm.MustandsforthesizeoftheGenepooland+1meansthateveryevolutionaryiterationonenewchildGenotypeisgeneratedfromparentsinthecurrentGenepool.

Thefollowingisexecutedeveryevolutionaryiteration:

- IncreasethesizeofGenepool(µ)by1:

o Selectparentsthroughexecutingatournamentselectionforeachparent.ParentsmustbedifferentGenotypes.

o Performcrossoveronthetwogenotypes:thereis50%chanceforeveryGeneofeachparent’sGenotypetoenduponthechildGenotype.

o EverygeneofthechildGenotypehasachanceofPmutateGenetogetmutated.Theamountofmutationiscomputedinthefollowingway:

newGeneValue=currentGeneValue*mutationFactor

mutationFactoristhepercentageofchangeappliedtotheGeneandlieswithintheintervalofminMutationFactorandmaxMutationFactor.

- TestthenewGenotypeandobtainafitnessvalue.

- BringthesizeofGenepoolbacktoµbyremovingtheGenotypewithlowestfitnessfromtheGenepool.

18

3.Tester

Tester=<Generator,Configurator,Evaluator>

TesterlinksEvolutionandEnvironmenttogether.Itexecutesiterationsinbothandrelaysgenotypicinformationbetweenthem.Theapproach(clonalora‐clonal)influencesthewayteamsaregeneratedandtested.IndependentfromthecurrentapproachConfiguratordefinestheconfigurationsinwhichteamsaretested.

3.1Generator

Twoapproaches,clonalanda‐clonal,differinhowteamsaregeneratedfromanuntestedGenotype.TheuntestedGenotypeisobtainedfromEvolution.

- Clonalapproach:createteamwithNAclonesoftheuntestedGenotype.

- A‐clonalapproach:createNaclonalTeamsPerGenotypenumberofa‐clonalteamsinsuchawaythatnoGenotypeisrepresentedinmorethanoneteam,withtheexceptionoftheuntestedGenotypethatisrepresentedonceineveryteam.

3.2Configurator

Togetagoodperformancemeasureandminimizingtheinfluenceofcoincidencemultiplestartingconfigurationsarespecified.

- Nconfigurationisthenumberofconfigurationsspecifiedforwhicheachteamistested.

- Everyconfigurationdefines:

o NSource,NGoalandNAgentthatdenotethenumberofSources,GoalsandAgentsinateamrespectively.

o StartLocation,orientationandvelocityofAgents

o StartLocationofSourcesandGoals.

3.3Evaluator

GeneratedteamsareevaluatedintheEnvironment.Afterapredefinedtimethetestendsandthefitnessvalueisrecorded.Thenumberoftestsperteamdependsontheapproachandthenumberofconfigurationsspecified.Tofurthereliminatetheinfluenceofcoincidenceteamsaretestedanumberoftimesperconfiguration.

- Testteaminconfigurations:

o Clonal:

19

TestteamforeveryconfigurationspecifiedbyTaskanumberoftimesdefinedbyNtestConfigurationClonally.

Numberoftests=numberoftaskconfigurations*NtestConfigurationClonally

o A‐clonal:

TestteamforeveryconfigurationspecifiedbyTaskanumberoftimesdefinedbyNtestConfigurationA‐ClonallyfromeachofthedifferentAgentstartingconfigurations.

Numberoftests=numberoftaskconfigurations*numberofAgentstartingconfigurationspertaskconfiguration*NtestConfigurationA‐Clonally

o ThenumberofteststhatisexecutedtocalculatetheGenotype’sfitnessthusdependsheavilyontheapproachthatisused.

- AnaveragefitnessvalueiscalculatedfromtheobtainedfitnessvaluesofeachofthetestsandreportedbacktotheEvolution.

20

AppendixB:ParametersettingsThefollowingtableshowstheparametersettingsastheywereusedintheexperiments.SeeAppendixAforparameterdefinitions.

Type Parameter Value

Package NpackagesAllowedInWorld 1

Source Nsource 2

Source NmaxPackagesAllowedInWorld 1

Source TmaxWaitingTimeToOfferPackage 300

Goal Ngoal 2

Agent NA 2

Agent minVelocityA 1

Agent maxVelocityA 4

Agent minEnergyA 4

Agent maxEnergyA 1

Agent energyStepA 0.01

Agent collisionTransferDelay 50

Task TtimeToTestTeam 1000

Evolution Ngenotype 30

Evolution NgenesPerGenotype/NagentWeights 7

Evolution PmutateGene 0.2

Evolution minMutationFactor 0

Evolution maxMutationFactor 0.1

Tester Nconfiguration 3

Tester NtestConfigurationClonally 2

Tester NtestConfigurationA‐Clonally 4