Upload
hester-newman
View
218
Download
2
Tags:
Embed Size (px)
Citation preview
Slide 1
Real-Time Neuroevolution in the NERO Video GameMike Taks Bram van de KlundertAboutPublished 2005Cited 286 timesKenneth O. StanleyAssociate Professor at University of Central FloridaRisto MiikkulainenProfessor at the University of Texas at AustinBobby D. BryantAssistant Professor at university of NevadaKenneth O. Stanley created NEAT, also recomened NERO project2ContentsIntroductionNEATrtNEATNEROIntroductionreal-time NeuroEvolution of Augmenting Topologies (rtNEAT)Adaption of the NEAT algorithmCreate new genre of games requiring learningBlack and whiteTamagotchi
NEATNeuro Evolution of Augmenting TopologiesGrowing neural network
RepresentationList of connection genesInnovation numberGlobal counterIn nodeOut nodeWeightEnabled
Initial populationUniform population of simple networksNo hidden nodesRandom weightsMutationWeight mutationStructure mutationAdd a connection between two nodesReplace a connection by a nodeConnection not removed only disabledOut connection inherits the valueCrossover termsDisjoint: gene is only in one networkExcess: disjoint and outside of the range of innovations
Crossovershared genes:Uniform crossoverBlend crossoverDisjoint and excess genesTaken from most fit parentCrossover exampleEqual fitness9, 10 excess6, 7 , 8 disjoint
SpeciationSpeciationSpecies assignmentCheck if there is a species close enough to the individualIf not, create new species FitnessSelectionTrailer NEROrtNEATDifferencesSelection and replacementRemoving agentDifferencesWork real timeOriginally NEAT evaluates one complete generation of individuals, generates offspring en masseDifferencesDuring a game, performance statistics are being recordedReplacing agentsPerform actions every n game-ticks
Selection and replacementCalculate fitnessRemove worst agent of sufficient ageChoose parents among the bestCreate offspring, Reassign all agents to species
Removing agentRemove worst agent based fitness adjusted for species sizeNew agents are continuously born, life time individually kept track ofPossibility to just replace the neural network of an agentNEROPlayer is a trainerSet up exercisesSave and load neural networks
Training modePlace objects on the field (static enemies, turrets, rovers, flags, ...)Adjust fitness rewards by sliders
Training modeAgents spawn in the factory
SensorsRadar to track enemy locationRangefinderLine-of-fire...
Evolved topology
Battle modeAssemble team of 20 agentsEnds if one team is empty
ExperimentsSlightly nondeterministic game engineThe same game is thus never played twice
BehaviorsDifferent behaviors are trainedSeek and fire by placing a single static enemy on the training fieldFiring and hitting a target was to slow to evolve. Aiming script was usedBehaviorsAvoidance trained by controlling an agent manuallyAgent runs backwards facing the enemy and shooting at it
BehaviorsTrain agents to avoid turret fire
More complex behaviorsLet agents attack enemy behind a wallTrain agents to avoid hazardous corridors
More complex behaviorsTrain agents versus targets that are standing against a wall
More complex behaviorsIncrementally add walls, agents will be able to navigate
BattlingPaper rock scissorsSeek vs avoidance
Battling36Questions?