The Framsticks systemversatile simulator of 3Dagents and their evolution

Maciej KomosinAcircskiInstitute of Computing Science Poznan University of Technology

Poznan Poland

Keywords Cybernetics Simulation Complexity

Abstract Various aspects of the Framsticks system are described The system is a universal toolfor modeling simulating and optimizing virtual agents with three-dimensional body andembedded control system Simulation model is described regrst Then features of the systemframework are presented with typical and potential applications Speciregc tools supporting humanunderstanding of evolutionary processes control and behaviors are also outlined Finally the mostinteresting research experiments are summarized

1 IntroductionLife is one of the most complex phenomena known in our world Researchersconstruct various models of life which serve diverse purposes ranging fromentertainment to medicine One of such models is Framsticks a systemdeveloped since 1997 (KomosinAcircski and Rotaru-Varga 2000 2001 KomosinAcircskiand Ulatowski 1997) It is different from most other models in that it does notaddress a single purpose or a single research problem On the contrary it isbuilt to support a wide range of experiments and to provide all of itsfunctionality to users who may use this open system in a variety of ways

Another feature which is special for Framsticks is the signiregcance ofunderstanding which is central for system development Although this is amuch simplireged model of reality it is easily capable of producing morecomplex phenomena than a human can comprehend (KomosinAcircski 2000) Thusit is essential to provide as many automatic analysissupport tools as it ispossible Intelligible visualization is one of the most fundamental means forhuman understanding of artiregcial life forms and this feature is certainlypresent in the software

The system is designed so that it does not introduce restrictions concerningcomplexity and size of creatures Thus neural networks can have any topology

MaciejKomosinskicsputpoznanpl wwwframsalifeplThis work has been supported by the State Committee for Scientiregc Research from KBNresearch grant no 8T11F 006 19 and by the Foundation for Polish Science from subsidyno 112001

The Emerald Research Register for this journal is available at The current issue and full text archive of this journal is available at

httpwwwemeraldinsightcomresearchregister httpwwwemeraldinsightcom0368-492Xhtm

K3212

156

KybernetesVol 32 No 12 2003pp 156-173

q MCB UP Limited0368-492XDOI 10110803684920310452382

and dimension allowing for a range of complex behaviors some described inKomosinAcircski (2000) section 4 Avoiding limitations is important becauseFramsticks is ultimately destined to simulation of open-ended evolution whereinteractions between creatures and environment are the sources of competitioncooperation communication intelligence etc

Further sections focus on the following issues Section 2 plusmn simulation(morphology control system environment) Section 3 plusmn general systemframework (genotype-phenotype relationship and possible usage of thesystem) Section 4 plusmn tools that the system provides to support research Section5 plusmn sample experiments which have already been performed as well as someideas for the future Section 6 summarizes this paper

2 SimulationFramsticks simulates a three-dimensional world and creatures All kinds ofinteraction between physical objects are considered static and dynamicfriction damping action and reaction forces energy losses after deformationsgravitation and uplift pressure plusmn buoyancy (in a water environment)

There is always a tradeoff between simulation accuracy and simulationtime We need a fast simulation to perform evolution on the other hand thesystem should be as realistic (detailed) as possible to produce realistic(complex) behaviors As we expect emergence of more and more sophisticatedphenomena the evolution has to be longer and the simulation must be lessaccurate but faster[1] Thus in order to make the simulation fast and due to thecomputational complexity some aspects like collisions between parts of anorganism itself were discarded

Artiregcial creatures in Framsticks are built of body and brain Body iscomposed of material points (called parts ) connected by elastic joints Brain ismade from neurons (these are signal processing units receptors and effectors)and neural connections For more detailed description of this model refer toGDK at Ulatowski (2000)

21 BodyThe basic element is a stick made of two macrexibly joined parts (regnite elementmethod is used for step-by-step simulation) Parts and joints have somefundamental properties like position orientation weight and friction butthere may also be other properties like the ability to assimilate energydurability of joints in collisions etc Articulations exist between sticks wherethey share an endpoint the articulations are unrestricted in all three degrees offreedom (bending in two planes plus twisting) Figure 1 shows forcesconsidered in the simulation

The Framstickssystem

157

22 BrainBrain (the control system) is made of neurons and their connections A neuronmay be a signal processing unit but it may also interact with body as areceptor or effector There are some prederegned types of neurons

The standard Framsticks neuron is more general than a popular weighted-sum sigmoid-transfer-function neuron used in AI three parameters are used foreach neuron They inmacruence speed and tendency of changes of the inner neuronstate and the steepness of the sigmoid transfer function However in thespecial case when the three parameters are assigned speciregc values thecharacteristics of a neuron becomes identical to the popular reactive AI neuronIn this case neural output remacrects instantly input signals More informationand sample neuronal runs can be found in the simulator details section atKomosinAcircski and Ulatowski (1997)

Another pre-deregned neuron is a sinus-generator with frequency controlledby its inputs Random noise generator neuron is also available It is easilypossible to add custom user-designed neurons

The neural network can have any topology and complexity Neurons can beconnected with each other in any way (some may be unconnected) Inputs canbe connected to outputs of other neurons (also senses) while outputs can beconnected to inputs of other neurons (also effectors plusmn muscles) A samplecontrol system is shown in Figure 2

23 Receptors and effectorsReceptors and effectors are interacting between body and brain They must beconnected to brain in order to be useful but they must also be a part of theworld Framsticks have currently three kinds of receptors (senses ) for

Figure 1Forces involved in thesimulation

K3212

158

orientation in space (equilibrium sense gyroscope) detection of physicalcontact (touch) and detection of energy (smell) See also Figure 2

Effectors (muscles) are placed on stick joints There are two kinds ofmuscles bending and rotating Positive and negative changes of muscle controlsignal make the sticks move in either direction plusmn it is analogous to the naturalsystems of muscles with macrexors and extensors The strength of a muscledetermines its effective ability of movement and speed (acceleration) Ifenergetic issues are to be considered then a stronger muscle consumes moreenergy during its work

A sample Framstick equipped with these elements is shown in Figure 3

24 EnvironmentThe world may be macrat build of smooth slopes or blocks (see Figure 4) It ispossible to adjust water level so that not only walkingrunningjumpingcreatures but also the swimming ones emerge World boundaries may be

none (the world is inregnite)

hard (fence it is impossible to cross the boundary)

wrap (crossing the boundary means teleportation to another world edge)

Figure 2A sample neural

network Triangles aresignal-processing

neurons You can seereceptors on the left

(gyroscope touchconstant signal) and

controlled muscles on theright (rotating bending)

Note recurrent andparallel connections

The Framstickssystem

159

All these options are useful in various kinds of experiments and performancemeasurements

3 Framework and evolution31 GeneticsThere are multiple genetic encodings (ordfgenotype languagesordm) supported by theFramsticks system each with its own representation and operators Thesystem manipulates and transforms genotype strings in variousrepresentations and ultimately decodes them into the internal representationused by the simulator

Any creature can be completely described using a low-level representationf 0 by listing all of its components and attributes This representation can betreated as a special genotype encoding plusmn special because it is a direct one-to-one mapping Other higher-level encodings convert their representation into thecorresponding f 0 version (possibly through another intermediaryrepresentation) The reverse mapping into higher-level encodings is difregcult

Figure 3Receptors (equilibriumtouch smell) andeffectors (muscles) inFramsticks

Figure 4Flat land smooth slopesand blocks

K3212

160

to compute which is also true for biological phenotype encodings As aconsequence in the general case it is not possible to convert a lower-levelrepresentation into a higher-level one (or a higher-level one into another higher-level one)

Each encoding has its associated genetic operators (mutation crossover andoptional repair) and a decoding procedure which translates a genotype into af 0 representation A new encoding can be added relatively easily byimplementing these components without the need to work with internalrepresentations The Framsticks system is accompanied by the GenotypeDevelopment Kit (GDK) to simplify this process (Ulatowski 2000) Later themost popular encodings are shortly described

The direct low-level or f 0 encoding describes agents exactly as they arerepresented in the simulator This encoding is more of a direct representationthan a proper encoding but it is possible to use it as such It does not use anyhigher-level features to make the genotype more compact or macrexible andbecause of this it is expected that this encoding is not very well suited forevolution Its useful characteristics are that it has a minimal decoding cost andthat it is universal every possible agent can be described using this encodingThese properties make it possible to use the f 0 encoding as an intermediaryrepresentation during the translation from other higher-level encodings

The direct low-level (f 0) genotype consists of a list of descriptions of all theobjects the agent is composed of parts joints neurons and connections Everydescription specireges all the attributes of the object explicitly Generally thisencoding does not impose any restriction on the phenotypes and it even allowsmorphologies with cycles

The recurrent direct encoding (f1) describes all the parts of thecorresponding phenotype Small changes in the genotype cause smallchanges in the resulting creature Body properties are represented locallybut propagate through a creaturersquos structure (with decreasing power) Thatmeans that most of the properties (and neural network connections) aremaintained when a part of a genotype is moved to another place Controlelements (neurons receptors) are associated with the elements under theircontrol (muscles sticks) Only treelike structures can be represented (no cyclesallowed) This encoding is relatively easy for humans to manipulate and designcreatures manually by editing their genotypes

The developmental encoding (f4) is development-oriented similar toencoding applied for evolving neural networks (Gruau et al 1996) Aninteresting merit of developmental encoding is that it can incorporate symmetryand modularity features commonly found in natural systems yet difregcult toformalize f4 is similar to f1 but codes are interpreted as commands by cells(sticks neurons etc) Cells can change their parameters and divide Each cellmaintains its own pointer to the current command in the genetic code Afterdivision cells can execute different codes and thus differentiate themselves

The Framstickssystem

161

The regnal body (phenotype) is the result of a development process it starts withan undifferentiated ancestor cell and ends with a collection of interconnecteddifferentiated cells (sticks neurons and connections)

Each of these encodings has its own genetic operators (mutation andcrossover) Each of them has been carefully designed and tested and eachencoding was based on numerous theoretical considerations More detaileddescription can be found in KomosinAcircski and Rotaru-Varga (2001) Examples ofsimple genotypes and corresponding phenotypes are shown in Figure 5

32 General frameworkThe most important feature of Framsticks is that you may deregne your ownrules for the simulator There are no predetermined laws but a script plusmnexperiment deregnition A script is a set of instructions in some language whichis interpreted by some program and executed

This script deregnes behavior of the Framsticks system in a few associatedareas

Creation of objects in the world The script deregnes where when and howmuch of what objects will be created An object is an evolved organismfood particle or any other element of the world designed by a researcherThus depending on some speciregc script food or obstacles might appearmove and disappear their location might depend on where creatures areetc

Objects interactions Object collisioncontact is an event which maycause some action deregned by the script author For example contact maymean energy ingestion pushing each other destruction or reproduction

Evolution A steady-state (one-at-a-time) selection model where a singlegenotype is inserted into a gene pool at a time can be used But a standard(ie generational replacement) evolutionary algorithm approach is alsopossible (a new gene pool replaces the whole old gene pool) Anotherpossibility is tournament competition for all pairs of genotypes Ingeneral the script can deregne many gene pools and many populations andperform independent evolutions under different conditions

Figure 5Left example of f 1genotypeXXX(XXX(XX))Right example of f4genotype with repetitiongene rr X 5 X RR llX LX LX X

K3212

162

Evaluation criteria are macrexible and do not have to be as simple as theperformances supplied by the simulator (but they will have to be basedsomehow on performances) For example regtness might depend on time orenergy required to fulregll some task or degree of success (distance fromtarget etc)

The script is built of ordfproceduresordm assigned to system events These include thefollowing events

onExpDefLoad plusmn occurs after experiment deregnition was loaded Thisprocedure should prepare the environment create gene pools andpopulations

onExpInit plusmn occurs at the beginning of the experiment

onExpSave plusmn occurs on save experiment request

onExpLoad plusmn occurs on load experiment request After this eventsystem state should resemble the state before onExpSave

onStep plusmn occurs in each simulation step

onBorn plusmn occurs when a new organism is created in the world

onKill plusmn occurs when a creature is removed from the world

on[X]Collision plusmn occurs when an object of group [X] has touchedsome other object

Thus a researcher may deregne the behavior of the whole system byimplementing appropriate actions within these events A single script(experiment deregnition) may use parameters so it usually allows to perform awhole bunch (class) of diversireged experiments

33 Illustrative example (standard experiment deregnition)The regle ordfstandardexpdefordm contains the full source for the script used tooptimize creatures on a steady-state basis with regtness deregned as a weightedsum of their performances (see also Figure 6) This script is quite versatile andcomplex Below its general idea is explained with much simplireged actionsassigned to system eventsonExpDefLoad

create single gene pool ordfGenotypesordm

create two populations ordfCreaturesordm and ordfFoodordm

onExpInit

empty all gene pools and populations

place the beginning genotype in ordfGenotypesordm

The Framstickssystem

163

onStep

if too little food create new object in ordfFoodordm

if too few organisms select a parent from ordfGenotypesordm mutate crossoveror copy it From the resulting genotype create an individual in ordfCreaturesordm

onBorn

move new object into a randomly chosen place in the world

set starting energy according to objectrsquos type

onKill

if ordfCreaturesordm object died save its performance in ordfGenotypesordm (possiblycreating a new genotype) If there are too many genotypes in ordfGenotypesordmremove one

onFoodCollision

send energy portion from ordfFoodordm object to ordfCreatureordm object

4 Additional toolsMany research works concern studies of evolutionary processes theirdynamics and efregciency Various measures and methods have been developedin order to be able to analyze evolution complexity and interaction in theobserved systems Other works try to understand behaviors of artiregcialcreatures regarding them as subjects of survey rather than black boxes withassigned regtness and performance

Artiregcial life systems especially those applied to evolutionary robotics anddesign (Bentley 1999 Funes and Pollack 1998 Lipson and Pollack 2000) arequite complex and it is difregcult to understand the behavior of existing agents in

Figure 6A fragment of softwarewindow with experimentparameters

K3212

164

detail The only way is to observe them carefully and use human intelligence todraw conclusions Usually the behavior of such agents is non-deterministicand their control systems are sophisticated often coupled with morphologyand very strongly connected functionally (Lund et al 1997)

Thus for the purpose of studying behaviors and populations of individualsone needs high-level intelligent support tools (KomosinAcircski and Rotaru-Varga2000) It is not likely that automatic tools will soon be able to produceunderstandable non-trivial explanations of sophisticated artiregcial agentsNonetheless their role and help cannot be ignored Even simple automaticsupport is of great relevance to a human which becomes obvious afterspending hours on investigating relatively simple artiregcial creatures It maybe that in the future some advanced analysis methods developed withinartiregcial life methodology will be useful for real life studies biology andmedicine

One of the main purposes of the Framsticks system is to allow creating andtesting such tools and procedures and to develop methodology needed fortheir use Realistic artiregcial life environments are the right place for suchresearch

41 Clustering of similar individualsSimilarity seems to be a simple property However automatic measures ofsimilarity can help in observation of regularities groups of related individualsetc Similarity can be identireged in many ways including aspects of morphology(body) brain size function behavior performance regtness etc Whateverderegnition is used automatic measure of similarity can be useful for

optimization to introduce artiregcial niches by modiregcation of regtnessvalues (Goldberg 1989)

studies of evolutionary processes and the structure of populations ofindividuals

studies of functionbehavior of agents

reduction of the number of agents to a small subset of interesting diverseunique individuals

inferring dendrograms (and hopefully phylogenetic trees) based ondistances between organisms

For Framsticks we constructed a heuristic method that is able to estimate thedegree of similarity of the two individuals This method treats body as a graph(with material points as vertices and joints as edges) and then tries to matchtwo body structures based on the degrees of vertices as the main piece ofinformation For a more detailed description of this method see KomosinAcircskiet al (2001) and an example is presented in Section 5

The Framstickssystem

165

42 History of evolutionIn real life although we are able to trace genetic relationships within existingcreatures we do not know what happens during mutation and crossing over oftheir genomes Moreover we cannot trace genetic relations in a longer scale andhigher number of individuals

In Framsticks it is possible not only to remember all parent-childrelationships but also to estimate genetic shares of related individuals (howmany genes have mutated or have been exchanged) This allows to draw a realtree of evolution as shown in Figure 7 Vertical axis is time horizontal oneremacrects local degree of genetic dissimilarity (between a pair of individuals)Vertices in the tree are single individuals

43 Fuzzy controlFuzzy control nowadays has become a very popular method of controlling inmany domains of our life washing machines video cameras ABS air-condition etc It is also used for controlling non-linear fast-changing processeswhere quick decisions are more important than exact ones (Yager and Filev1994) Fuzzy control

Figure 7The real tree ofevolution Top singleancestor and beginningof time Black linesrepresent mutationswhite ones arecrossovers

K3212

166

allows for linguistic variables ordfDrive fastordm is much more macrexible thanordfdrive with speed 80-100 kmhordm In fuzzy approach we would say 79 kmhis still fast with degree 95 percent

better comprehension for humans plusmn fuzzy rule if X is Big and Y is Smallthen Z is Medium is easier to understand than crispy one if X is between3222 and 4332 and Y is less than 52 then Z is 192

more macruent ordfnaturalordm change of system states plusmn for example gradualslowing down of a car

For such reasons fuzzy control is developed in Framsticks Therefore evolvedFramsticksrsquo brains may be more understandable for a human Fuzzy controlsimilarly to neural networks can also cope with uncertainty of information plusmnwhen the process is compound depends on lottery events or the measurementis burden with error Fuzzy approach can manage this inconvenience bygeneralization of information To incorporate this approach in Framsticksneural network model three additional types of neurons are needed

difference neuron plusmn to compare current value of the sensor to the previousone

rule neuron plusmn to represent a fuzzy rule

aggregate neuron plusmn to gather answers of the rule neurons and calculatecrisp output

The two sample fuzzy rules are

IF gyroscope is big+ AND Dgyroscope is small+

THEN bend muscle medium 2

IF gyroscope is medium 2 AND Dgyroscope is zero+

THEN bend muscle big+

where gyroscope is the measurement taken from the gyroscope receptorDgyroscope is the change in its value and linguistic terms ordfbig+ordm etccorrespond to fuzzy intervals of values The representation of these two rules inthe Framsticks neural network is shown in Figure 8

44 Helper softwareTogether with continuous development of extensions to the Framsticks systemitself there are some external tools created Fred is a visual Framsticks Editordeveloped in JAVA which allows to design a creature (or a structure) in a user-friendly way Framsticks Experimentation Center is an Internet database ofgenotypes and experiment deregnitionsproposals which can be browseddownloaded and uploaded

The Framstickssystem

167

5 Research experiments51 Comparison of genotype encodingsThere are a number of studies of the evolution of simulated creatures equippedwith realistic physical behavior In such systems the use of a physicalsimulation layer implements a complex genotype-regtness relationshipPhysical interactions between body parts the coupling between control andphysical body and interactions during body development can all add a level ofindirection between the genotype and regtness The complexity of the genotype-regtness relationship offers a potential for rich evolutionary dynamics

The most important element of the genotype-to-regtness relationship is thegenotype-to-phenotype mapping or genotype encoding There is no obvioussimple way to encode a complex phenotype plusmn which consists of a variable-sizestructured body and a matching control system plusmn into a simpler genotypeMoreover an evolutionary algorithm can perform poorly when using a certaingenotype encoding and better when using others for reasons not yetimmediately obvious The employed genotype encoding can have a signiregcanteffect on the performance of the evolution

The Framsticks system has been used as the context of analysis of variousgenotype encodings The performance of the three encodings described inSection 3 was compared in their optimization tasks passive and active heightand velocity maximization (KomosinAcircski and Rotaru-Varga 2001) Thesolutions produced by evolution are considered to be successful for the giventasks in all three cases However there were some important differences in thedegree of success The f 0 encoding performed worse than the two higher-level

Figure 8Two fuzzy rules encodedas a Framsticks neuralnetwork

K3212

168

encodings The most important differences between these encodings are thatf 0 has a minimal bias and is unrestrictive while the higher-level encodings (f1and f4) restrict the search space and introduce a strong bias towardsstructured phenotypes Based on these results we conclude that a morestructured genotype encoding with genetic operators working on a higherlevel is beneregcial in the evolution of 3D agents The presence of a bias towardsstructured phenotypes can overcome the apparent limitation that entire regionsof the search space are not accessible by the search This bias may be useful insome applications (engineering and robotics for example) The signiregcantinmacruence of a chosen encoding can be clearly seen in the obtained agents thosewith f 0 encoding displayed neither order nor structure The two encodingsrestricting morphology to a tree produced more clear constructions and fordevelopmental encoding segmentation and modularity could be observed(Figure 9)

Figure 9Representative agents

for various geneticencodings and height

maximization task

The Framstickssystem

169

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

22 BrainBrain (the control system) is made of neurons and their connections A neuronmay be a signal processing unit but it may also interact with body as areceptor or effector There are some prederegned types of neurons

The standard Framsticks neuron is more general than a popular weighted-sum sigmoid-transfer-function neuron used in AI three parameters are used foreach neuron They inmacruence speed and tendency of changes of the inner neuronstate and the steepness of the sigmoid transfer function However in thespecial case when the three parameters are assigned speciregc values thecharacteristics of a neuron becomes identical to the popular reactive AI neuronIn this case neural output remacrects instantly input signals More informationand sample neuronal runs can be found in the simulator details section atKomosinAcircski and Ulatowski (1997)

Another pre-deregned neuron is a sinus-generator with frequency controlledby its inputs Random noise generator neuron is also available It is easilypossible to add custom user-designed neurons

The neural network can have any topology and complexity Neurons can beconnected with each other in any way (some may be unconnected) Inputs canbe connected to outputs of other neurons (also senses) while outputs can beconnected to inputs of other neurons (also effectors plusmn muscles) A samplecontrol system is shown in Figure 2

23 Receptors and effectorsReceptors and effectors are interacting between body and brain They must beconnected to brain in order to be useful but they must also be a part of theworld Framsticks have currently three kinds of receptors (senses ) for

Figure 1Forces involved in thesimulation

K3212

158

orientation in space (equilibrium sense gyroscope) detection of physicalcontact (touch) and detection of energy (smell) See also Figure 2

Effectors (muscles) are placed on stick joints There are two kinds ofmuscles bending and rotating Positive and negative changes of muscle controlsignal make the sticks move in either direction plusmn it is analogous to the naturalsystems of muscles with macrexors and extensors The strength of a muscledetermines its effective ability of movement and speed (acceleration) Ifenergetic issues are to be considered then a stronger muscle consumes moreenergy during its work

A sample Framstick equipped with these elements is shown in Figure 3

24 EnvironmentThe world may be macrat build of smooth slopes or blocks (see Figure 4) It ispossible to adjust water level so that not only walkingrunningjumpingcreatures but also the swimming ones emerge World boundaries may be

none (the world is inregnite)

hard (fence it is impossible to cross the boundary)

wrap (crossing the boundary means teleportation to another world edge)

Figure 2A sample neural

network Triangles aresignal-processing

neurons You can seereceptors on the left

(gyroscope touchconstant signal) and

controlled muscles on theright (rotating bending)

Note recurrent andparallel connections

The Framstickssystem

159

All these options are useful in various kinds of experiments and performancemeasurements

3 Framework and evolution31 GeneticsThere are multiple genetic encodings (ordfgenotype languagesordm) supported by theFramsticks system each with its own representation and operators Thesystem manipulates and transforms genotype strings in variousrepresentations and ultimately decodes them into the internal representationused by the simulator

Any creature can be completely described using a low-level representationf 0 by listing all of its components and attributes This representation can betreated as a special genotype encoding plusmn special because it is a direct one-to-one mapping Other higher-level encodings convert their representation into thecorresponding f 0 version (possibly through another intermediaryrepresentation) The reverse mapping into higher-level encodings is difregcult

Figure 3Receptors (equilibriumtouch smell) andeffectors (muscles) inFramsticks

Figure 4Flat land smooth slopesand blocks

K3212

160

to compute which is also true for biological phenotype encodings As aconsequence in the general case it is not possible to convert a lower-levelrepresentation into a higher-level one (or a higher-level one into another higher-level one)

Each encoding has its associated genetic operators (mutation crossover andoptional repair) and a decoding procedure which translates a genotype into af 0 representation A new encoding can be added relatively easily byimplementing these components without the need to work with internalrepresentations The Framsticks system is accompanied by the GenotypeDevelopment Kit (GDK) to simplify this process (Ulatowski 2000) Later themost popular encodings are shortly described

The direct low-level or f 0 encoding describes agents exactly as they arerepresented in the simulator This encoding is more of a direct representationthan a proper encoding but it is possible to use it as such It does not use anyhigher-level features to make the genotype more compact or macrexible andbecause of this it is expected that this encoding is not very well suited forevolution Its useful characteristics are that it has a minimal decoding cost andthat it is universal every possible agent can be described using this encodingThese properties make it possible to use the f 0 encoding as an intermediaryrepresentation during the translation from other higher-level encodings

The direct low-level (f 0) genotype consists of a list of descriptions of all theobjects the agent is composed of parts joints neurons and connections Everydescription specireges all the attributes of the object explicitly Generally thisencoding does not impose any restriction on the phenotypes and it even allowsmorphologies with cycles

The recurrent direct encoding (f1) describes all the parts of thecorresponding phenotype Small changes in the genotype cause smallchanges in the resulting creature Body properties are represented locallybut propagate through a creaturersquos structure (with decreasing power) Thatmeans that most of the properties (and neural network connections) aremaintained when a part of a genotype is moved to another place Controlelements (neurons receptors) are associated with the elements under theircontrol (muscles sticks) Only treelike structures can be represented (no cyclesallowed) This encoding is relatively easy for humans to manipulate and designcreatures manually by editing their genotypes

The developmental encoding (f4) is development-oriented similar toencoding applied for evolving neural networks (Gruau et al 1996) Aninteresting merit of developmental encoding is that it can incorporate symmetryand modularity features commonly found in natural systems yet difregcult toformalize f4 is similar to f1 but codes are interpreted as commands by cells(sticks neurons etc) Cells can change their parameters and divide Each cellmaintains its own pointer to the current command in the genetic code Afterdivision cells can execute different codes and thus differentiate themselves

The Framstickssystem

161

The regnal body (phenotype) is the result of a development process it starts withan undifferentiated ancestor cell and ends with a collection of interconnecteddifferentiated cells (sticks neurons and connections)

Each of these encodings has its own genetic operators (mutation andcrossover) Each of them has been carefully designed and tested and eachencoding was based on numerous theoretical considerations More detaileddescription can be found in KomosinAcircski and Rotaru-Varga (2001) Examples ofsimple genotypes and corresponding phenotypes are shown in Figure 5

32 General frameworkThe most important feature of Framsticks is that you may deregne your ownrules for the simulator There are no predetermined laws but a script plusmnexperiment deregnition A script is a set of instructions in some language whichis interpreted by some program and executed

This script deregnes behavior of the Framsticks system in a few associatedareas

Creation of objects in the world The script deregnes where when and howmuch of what objects will be created An object is an evolved organismfood particle or any other element of the world designed by a researcherThus depending on some speciregc script food or obstacles might appearmove and disappear their location might depend on where creatures areetc

Objects interactions Object collisioncontact is an event which maycause some action deregned by the script author For example contact maymean energy ingestion pushing each other destruction or reproduction

Evolution A steady-state (one-at-a-time) selection model where a singlegenotype is inserted into a gene pool at a time can be used But a standard(ie generational replacement) evolutionary algorithm approach is alsopossible (a new gene pool replaces the whole old gene pool) Anotherpossibility is tournament competition for all pairs of genotypes Ingeneral the script can deregne many gene pools and many populations andperform independent evolutions under different conditions

Figure 5Left example of f 1genotypeXXX(XXX(XX))Right example of f4genotype with repetitiongene rr X 5 X RR llX LX LX X

K3212

162

Evaluation criteria are macrexible and do not have to be as simple as theperformances supplied by the simulator (but they will have to be basedsomehow on performances) For example regtness might depend on time orenergy required to fulregll some task or degree of success (distance fromtarget etc)

The script is built of ordfproceduresordm assigned to system events These include thefollowing events

onExpDefLoad plusmn occurs after experiment deregnition was loaded Thisprocedure should prepare the environment create gene pools andpopulations

onExpInit plusmn occurs at the beginning of the experiment

onExpSave plusmn occurs on save experiment request

onExpLoad plusmn occurs on load experiment request After this eventsystem state should resemble the state before onExpSave

onStep plusmn occurs in each simulation step

onBorn plusmn occurs when a new organism is created in the world

onKill plusmn occurs when a creature is removed from the world

on[X]Collision plusmn occurs when an object of group [X] has touchedsome other object

Thus a researcher may deregne the behavior of the whole system byimplementing appropriate actions within these events A single script(experiment deregnition) may use parameters so it usually allows to perform awhole bunch (class) of diversireged experiments

33 Illustrative example (standard experiment deregnition)The regle ordfstandardexpdefordm contains the full source for the script used tooptimize creatures on a steady-state basis with regtness deregned as a weightedsum of their performances (see also Figure 6) This script is quite versatile andcomplex Below its general idea is explained with much simplireged actionsassigned to system eventsonExpDefLoad

create single gene pool ordfGenotypesordm

create two populations ordfCreaturesordm and ordfFoodordm

onExpInit

empty all gene pools and populations

place the beginning genotype in ordfGenotypesordm

The Framstickssystem

163

onStep

if too little food create new object in ordfFoodordm

if too few organisms select a parent from ordfGenotypesordm mutate crossoveror copy it From the resulting genotype create an individual in ordfCreaturesordm

onBorn

move new object into a randomly chosen place in the world

set starting energy according to objectrsquos type

onKill

if ordfCreaturesordm object died save its performance in ordfGenotypesordm (possiblycreating a new genotype) If there are too many genotypes in ordfGenotypesordmremove one

onFoodCollision

send energy portion from ordfFoodordm object to ordfCreatureordm object

4 Additional toolsMany research works concern studies of evolutionary processes theirdynamics and efregciency Various measures and methods have been developedin order to be able to analyze evolution complexity and interaction in theobserved systems Other works try to understand behaviors of artiregcialcreatures regarding them as subjects of survey rather than black boxes withassigned regtness and performance

Artiregcial life systems especially those applied to evolutionary robotics anddesign (Bentley 1999 Funes and Pollack 1998 Lipson and Pollack 2000) arequite complex and it is difregcult to understand the behavior of existing agents in

Figure 6A fragment of softwarewindow with experimentparameters

K3212

164

detail The only way is to observe them carefully and use human intelligence todraw conclusions Usually the behavior of such agents is non-deterministicand their control systems are sophisticated often coupled with morphologyand very strongly connected functionally (Lund et al 1997)

Thus for the purpose of studying behaviors and populations of individualsone needs high-level intelligent support tools (KomosinAcircski and Rotaru-Varga2000) It is not likely that automatic tools will soon be able to produceunderstandable non-trivial explanations of sophisticated artiregcial agentsNonetheless their role and help cannot be ignored Even simple automaticsupport is of great relevance to a human which becomes obvious afterspending hours on investigating relatively simple artiregcial creatures It maybe that in the future some advanced analysis methods developed withinartiregcial life methodology will be useful for real life studies biology andmedicine

One of the main purposes of the Framsticks system is to allow creating andtesting such tools and procedures and to develop methodology needed fortheir use Realistic artiregcial life environments are the right place for suchresearch

41 Clustering of similar individualsSimilarity seems to be a simple property However automatic measures ofsimilarity can help in observation of regularities groups of related individualsetc Similarity can be identireged in many ways including aspects of morphology(body) brain size function behavior performance regtness etc Whateverderegnition is used automatic measure of similarity can be useful for

optimization to introduce artiregcial niches by modiregcation of regtnessvalues (Goldberg 1989)

studies of evolutionary processes and the structure of populations ofindividuals

studies of functionbehavior of agents

reduction of the number of agents to a small subset of interesting diverseunique individuals

inferring dendrograms (and hopefully phylogenetic trees) based ondistances between organisms

For Framsticks we constructed a heuristic method that is able to estimate thedegree of similarity of the two individuals This method treats body as a graph(with material points as vertices and joints as edges) and then tries to matchtwo body structures based on the degrees of vertices as the main piece ofinformation For a more detailed description of this method see KomosinAcircskiet al (2001) and an example is presented in Section 5

The Framstickssystem

165

42 History of evolutionIn real life although we are able to trace genetic relationships within existingcreatures we do not know what happens during mutation and crossing over oftheir genomes Moreover we cannot trace genetic relations in a longer scale andhigher number of individuals

In Framsticks it is possible not only to remember all parent-childrelationships but also to estimate genetic shares of related individuals (howmany genes have mutated or have been exchanged) This allows to draw a realtree of evolution as shown in Figure 7 Vertical axis is time horizontal oneremacrects local degree of genetic dissimilarity (between a pair of individuals)Vertices in the tree are single individuals

43 Fuzzy controlFuzzy control nowadays has become a very popular method of controlling inmany domains of our life washing machines video cameras ABS air-condition etc It is also used for controlling non-linear fast-changing processeswhere quick decisions are more important than exact ones (Yager and Filev1994) Fuzzy control

Figure 7The real tree ofevolution Top singleancestor and beginningof time Black linesrepresent mutationswhite ones arecrossovers

K3212

166

allows for linguistic variables ordfDrive fastordm is much more macrexible thanordfdrive with speed 80-100 kmhordm In fuzzy approach we would say 79 kmhis still fast with degree 95 percent

better comprehension for humans plusmn fuzzy rule if X is Big and Y is Smallthen Z is Medium is easier to understand than crispy one if X is between3222 and 4332 and Y is less than 52 then Z is 192

more macruent ordfnaturalordm change of system states plusmn for example gradualslowing down of a car

For such reasons fuzzy control is developed in Framsticks Therefore evolvedFramsticksrsquo brains may be more understandable for a human Fuzzy controlsimilarly to neural networks can also cope with uncertainty of information plusmnwhen the process is compound depends on lottery events or the measurementis burden with error Fuzzy approach can manage this inconvenience bygeneralization of information To incorporate this approach in Framsticksneural network model three additional types of neurons are needed

difference neuron plusmn to compare current value of the sensor to the previousone

rule neuron plusmn to represent a fuzzy rule

aggregate neuron plusmn to gather answers of the rule neurons and calculatecrisp output

The two sample fuzzy rules are

IF gyroscope is big+ AND Dgyroscope is small+

THEN bend muscle medium 2

IF gyroscope is medium 2 AND Dgyroscope is zero+

THEN bend muscle big+

where gyroscope is the measurement taken from the gyroscope receptorDgyroscope is the change in its value and linguistic terms ordfbig+ordm etccorrespond to fuzzy intervals of values The representation of these two rules inthe Framsticks neural network is shown in Figure 8

44 Helper softwareTogether with continuous development of extensions to the Framsticks systemitself there are some external tools created Fred is a visual Framsticks Editordeveloped in JAVA which allows to design a creature (or a structure) in a user-friendly way Framsticks Experimentation Center is an Internet database ofgenotypes and experiment deregnitionsproposals which can be browseddownloaded and uploaded

The Framstickssystem

167

5 Research experiments51 Comparison of genotype encodingsThere are a number of studies of the evolution of simulated creatures equippedwith realistic physical behavior In such systems the use of a physicalsimulation layer implements a complex genotype-regtness relationshipPhysical interactions between body parts the coupling between control andphysical body and interactions during body development can all add a level ofindirection between the genotype and regtness The complexity of the genotype-regtness relationship offers a potential for rich evolutionary dynamics

The most important element of the genotype-to-regtness relationship is thegenotype-to-phenotype mapping or genotype encoding There is no obvioussimple way to encode a complex phenotype plusmn which consists of a variable-sizestructured body and a matching control system plusmn into a simpler genotypeMoreover an evolutionary algorithm can perform poorly when using a certaingenotype encoding and better when using others for reasons not yetimmediately obvious The employed genotype encoding can have a signiregcanteffect on the performance of the evolution

The Framsticks system has been used as the context of analysis of variousgenotype encodings The performance of the three encodings described inSection 3 was compared in their optimization tasks passive and active heightand velocity maximization (KomosinAcircski and Rotaru-Varga 2001) Thesolutions produced by evolution are considered to be successful for the giventasks in all three cases However there were some important differences in thedegree of success The f 0 encoding performed worse than the two higher-level

Figure 8Two fuzzy rules encodedas a Framsticks neuralnetwork

K3212

168

encodings The most important differences between these encodings are thatf 0 has a minimal bias and is unrestrictive while the higher-level encodings (f1and f4) restrict the search space and introduce a strong bias towardsstructured phenotypes Based on these results we conclude that a morestructured genotype encoding with genetic operators working on a higherlevel is beneregcial in the evolution of 3D agents The presence of a bias towardsstructured phenotypes can overcome the apparent limitation that entire regionsof the search space are not accessible by the search This bias may be useful insome applications (engineering and robotics for example) The signiregcantinmacruence of a chosen encoding can be clearly seen in the obtained agents thosewith f 0 encoding displayed neither order nor structure The two encodingsrestricting morphology to a tree produced more clear constructions and fordevelopmental encoding segmentation and modularity could be observed(Figure 9)

Figure 9Representative agents

for various geneticencodings and height

maximization task

The Framstickssystem

169

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

orientation in space (equilibrium sense gyroscope) detection of physicalcontact (touch) and detection of energy (smell) See also Figure 2

Effectors (muscles) are placed on stick joints There are two kinds ofmuscles bending and rotating Positive and negative changes of muscle controlsignal make the sticks move in either direction plusmn it is analogous to the naturalsystems of muscles with macrexors and extensors The strength of a muscledetermines its effective ability of movement and speed (acceleration) Ifenergetic issues are to be considered then a stronger muscle consumes moreenergy during its work

A sample Framstick equipped with these elements is shown in Figure 3

24 EnvironmentThe world may be macrat build of smooth slopes or blocks (see Figure 4) It ispossible to adjust water level so that not only walkingrunningjumpingcreatures but also the swimming ones emerge World boundaries may be

none (the world is inregnite)

hard (fence it is impossible to cross the boundary)

wrap (crossing the boundary means teleportation to another world edge)

Figure 2A sample neural

network Triangles aresignal-processing

neurons You can seereceptors on the left

(gyroscope touchconstant signal) and

controlled muscles on theright (rotating bending)

Note recurrent andparallel connections

The Framstickssystem

159

All these options are useful in various kinds of experiments and performancemeasurements

3 Framework and evolution31 GeneticsThere are multiple genetic encodings (ordfgenotype languagesordm) supported by theFramsticks system each with its own representation and operators Thesystem manipulates and transforms genotype strings in variousrepresentations and ultimately decodes them into the internal representationused by the simulator

Any creature can be completely described using a low-level representationf 0 by listing all of its components and attributes This representation can betreated as a special genotype encoding plusmn special because it is a direct one-to-one mapping Other higher-level encodings convert their representation into thecorresponding f 0 version (possibly through another intermediaryrepresentation) The reverse mapping into higher-level encodings is difregcult

Figure 3Receptors (equilibriumtouch smell) andeffectors (muscles) inFramsticks

Figure 4Flat land smooth slopesand blocks

K3212

160

to compute which is also true for biological phenotype encodings As aconsequence in the general case it is not possible to convert a lower-levelrepresentation into a higher-level one (or a higher-level one into another higher-level one)

Each encoding has its associated genetic operators (mutation crossover andoptional repair) and a decoding procedure which translates a genotype into af 0 representation A new encoding can be added relatively easily byimplementing these components without the need to work with internalrepresentations The Framsticks system is accompanied by the GenotypeDevelopment Kit (GDK) to simplify this process (Ulatowski 2000) Later themost popular encodings are shortly described

The direct low-level or f 0 encoding describes agents exactly as they arerepresented in the simulator This encoding is more of a direct representationthan a proper encoding but it is possible to use it as such It does not use anyhigher-level features to make the genotype more compact or macrexible andbecause of this it is expected that this encoding is not very well suited forevolution Its useful characteristics are that it has a minimal decoding cost andthat it is universal every possible agent can be described using this encodingThese properties make it possible to use the f 0 encoding as an intermediaryrepresentation during the translation from other higher-level encodings

The direct low-level (f 0) genotype consists of a list of descriptions of all theobjects the agent is composed of parts joints neurons and connections Everydescription specireges all the attributes of the object explicitly Generally thisencoding does not impose any restriction on the phenotypes and it even allowsmorphologies with cycles

The recurrent direct encoding (f1) describes all the parts of thecorresponding phenotype Small changes in the genotype cause smallchanges in the resulting creature Body properties are represented locallybut propagate through a creaturersquos structure (with decreasing power) Thatmeans that most of the properties (and neural network connections) aremaintained when a part of a genotype is moved to another place Controlelements (neurons receptors) are associated with the elements under theircontrol (muscles sticks) Only treelike structures can be represented (no cyclesallowed) This encoding is relatively easy for humans to manipulate and designcreatures manually by editing their genotypes

The developmental encoding (f4) is development-oriented similar toencoding applied for evolving neural networks (Gruau et al 1996) Aninteresting merit of developmental encoding is that it can incorporate symmetryand modularity features commonly found in natural systems yet difregcult toformalize f4 is similar to f1 but codes are interpreted as commands by cells(sticks neurons etc) Cells can change their parameters and divide Each cellmaintains its own pointer to the current command in the genetic code Afterdivision cells can execute different codes and thus differentiate themselves

The Framstickssystem

161

The regnal body (phenotype) is the result of a development process it starts withan undifferentiated ancestor cell and ends with a collection of interconnecteddifferentiated cells (sticks neurons and connections)

Each of these encodings has its own genetic operators (mutation andcrossover) Each of them has been carefully designed and tested and eachencoding was based on numerous theoretical considerations More detaileddescription can be found in KomosinAcircski and Rotaru-Varga (2001) Examples ofsimple genotypes and corresponding phenotypes are shown in Figure 5

32 General frameworkThe most important feature of Framsticks is that you may deregne your ownrules for the simulator There are no predetermined laws but a script plusmnexperiment deregnition A script is a set of instructions in some language whichis interpreted by some program and executed

This script deregnes behavior of the Framsticks system in a few associatedareas

Creation of objects in the world The script deregnes where when and howmuch of what objects will be created An object is an evolved organismfood particle or any other element of the world designed by a researcherThus depending on some speciregc script food or obstacles might appearmove and disappear their location might depend on where creatures areetc

Objects interactions Object collisioncontact is an event which maycause some action deregned by the script author For example contact maymean energy ingestion pushing each other destruction or reproduction

Evolution A steady-state (one-at-a-time) selection model where a singlegenotype is inserted into a gene pool at a time can be used But a standard(ie generational replacement) evolutionary algorithm approach is alsopossible (a new gene pool replaces the whole old gene pool) Anotherpossibility is tournament competition for all pairs of genotypes Ingeneral the script can deregne many gene pools and many populations andperform independent evolutions under different conditions

Figure 5Left example of f 1genotypeXXX(XXX(XX))Right example of f4genotype with repetitiongene rr X 5 X RR llX LX LX X

K3212

162

Evaluation criteria are macrexible and do not have to be as simple as theperformances supplied by the simulator (but they will have to be basedsomehow on performances) For example regtness might depend on time orenergy required to fulregll some task or degree of success (distance fromtarget etc)

The script is built of ordfproceduresordm assigned to system events These include thefollowing events

onExpDefLoad plusmn occurs after experiment deregnition was loaded Thisprocedure should prepare the environment create gene pools andpopulations

onExpInit plusmn occurs at the beginning of the experiment

onExpSave plusmn occurs on save experiment request

onExpLoad plusmn occurs on load experiment request After this eventsystem state should resemble the state before onExpSave

onStep plusmn occurs in each simulation step

onBorn plusmn occurs when a new organism is created in the world

onKill plusmn occurs when a creature is removed from the world

on[X]Collision plusmn occurs when an object of group [X] has touchedsome other object

Thus a researcher may deregne the behavior of the whole system byimplementing appropriate actions within these events A single script(experiment deregnition) may use parameters so it usually allows to perform awhole bunch (class) of diversireged experiments

33 Illustrative example (standard experiment deregnition)The regle ordfstandardexpdefordm contains the full source for the script used tooptimize creatures on a steady-state basis with regtness deregned as a weightedsum of their performances (see also Figure 6) This script is quite versatile andcomplex Below its general idea is explained with much simplireged actionsassigned to system eventsonExpDefLoad

create single gene pool ordfGenotypesordm

create two populations ordfCreaturesordm and ordfFoodordm

onExpInit

empty all gene pools and populations

place the beginning genotype in ordfGenotypesordm

The Framstickssystem

163

onStep

if too little food create new object in ordfFoodordm

if too few organisms select a parent from ordfGenotypesordm mutate crossoveror copy it From the resulting genotype create an individual in ordfCreaturesordm

onBorn

move new object into a randomly chosen place in the world

set starting energy according to objectrsquos type

onKill

if ordfCreaturesordm object died save its performance in ordfGenotypesordm (possiblycreating a new genotype) If there are too many genotypes in ordfGenotypesordmremove one

onFoodCollision

send energy portion from ordfFoodordm object to ordfCreatureordm object

4 Additional toolsMany research works concern studies of evolutionary processes theirdynamics and efregciency Various measures and methods have been developedin order to be able to analyze evolution complexity and interaction in theobserved systems Other works try to understand behaviors of artiregcialcreatures regarding them as subjects of survey rather than black boxes withassigned regtness and performance

Artiregcial life systems especially those applied to evolutionary robotics anddesign (Bentley 1999 Funes and Pollack 1998 Lipson and Pollack 2000) arequite complex and it is difregcult to understand the behavior of existing agents in

Figure 6A fragment of softwarewindow with experimentparameters

K3212

164

detail The only way is to observe them carefully and use human intelligence todraw conclusions Usually the behavior of such agents is non-deterministicand their control systems are sophisticated often coupled with morphologyand very strongly connected functionally (Lund et al 1997)

Thus for the purpose of studying behaviors and populations of individualsone needs high-level intelligent support tools (KomosinAcircski and Rotaru-Varga2000) It is not likely that automatic tools will soon be able to produceunderstandable non-trivial explanations of sophisticated artiregcial agentsNonetheless their role and help cannot be ignored Even simple automaticsupport is of great relevance to a human which becomes obvious afterspending hours on investigating relatively simple artiregcial creatures It maybe that in the future some advanced analysis methods developed withinartiregcial life methodology will be useful for real life studies biology andmedicine

One of the main purposes of the Framsticks system is to allow creating andtesting such tools and procedures and to develop methodology needed fortheir use Realistic artiregcial life environments are the right place for suchresearch

41 Clustering of similar individualsSimilarity seems to be a simple property However automatic measures ofsimilarity can help in observation of regularities groups of related individualsetc Similarity can be identireged in many ways including aspects of morphology(body) brain size function behavior performance regtness etc Whateverderegnition is used automatic measure of similarity can be useful for

optimization to introduce artiregcial niches by modiregcation of regtnessvalues (Goldberg 1989)

studies of evolutionary processes and the structure of populations ofindividuals

studies of functionbehavior of agents

reduction of the number of agents to a small subset of interesting diverseunique individuals

inferring dendrograms (and hopefully phylogenetic trees) based ondistances between organisms

For Framsticks we constructed a heuristic method that is able to estimate thedegree of similarity of the two individuals This method treats body as a graph(with material points as vertices and joints as edges) and then tries to matchtwo body structures based on the degrees of vertices as the main piece ofinformation For a more detailed description of this method see KomosinAcircskiet al (2001) and an example is presented in Section 5

The Framstickssystem

165

42 History of evolutionIn real life although we are able to trace genetic relationships within existingcreatures we do not know what happens during mutation and crossing over oftheir genomes Moreover we cannot trace genetic relations in a longer scale andhigher number of individuals

In Framsticks it is possible not only to remember all parent-childrelationships but also to estimate genetic shares of related individuals (howmany genes have mutated or have been exchanged) This allows to draw a realtree of evolution as shown in Figure 7 Vertical axis is time horizontal oneremacrects local degree of genetic dissimilarity (between a pair of individuals)Vertices in the tree are single individuals

43 Fuzzy controlFuzzy control nowadays has become a very popular method of controlling inmany domains of our life washing machines video cameras ABS air-condition etc It is also used for controlling non-linear fast-changing processeswhere quick decisions are more important than exact ones (Yager and Filev1994) Fuzzy control

Figure 7The real tree ofevolution Top singleancestor and beginningof time Black linesrepresent mutationswhite ones arecrossovers

K3212

166

allows for linguistic variables ordfDrive fastordm is much more macrexible thanordfdrive with speed 80-100 kmhordm In fuzzy approach we would say 79 kmhis still fast with degree 95 percent

better comprehension for humans plusmn fuzzy rule if X is Big and Y is Smallthen Z is Medium is easier to understand than crispy one if X is between3222 and 4332 and Y is less than 52 then Z is 192

more macruent ordfnaturalordm change of system states plusmn for example gradualslowing down of a car

For such reasons fuzzy control is developed in Framsticks Therefore evolvedFramsticksrsquo brains may be more understandable for a human Fuzzy controlsimilarly to neural networks can also cope with uncertainty of information plusmnwhen the process is compound depends on lottery events or the measurementis burden with error Fuzzy approach can manage this inconvenience bygeneralization of information To incorporate this approach in Framsticksneural network model three additional types of neurons are needed

difference neuron plusmn to compare current value of the sensor to the previousone

rule neuron plusmn to represent a fuzzy rule

aggregate neuron plusmn to gather answers of the rule neurons and calculatecrisp output

The two sample fuzzy rules are

IF gyroscope is big+ AND Dgyroscope is small+

THEN bend muscle medium 2

IF gyroscope is medium 2 AND Dgyroscope is zero+

THEN bend muscle big+

where gyroscope is the measurement taken from the gyroscope receptorDgyroscope is the change in its value and linguistic terms ordfbig+ordm etccorrespond to fuzzy intervals of values The representation of these two rules inthe Framsticks neural network is shown in Figure 8

44 Helper softwareTogether with continuous development of extensions to the Framsticks systemitself there are some external tools created Fred is a visual Framsticks Editordeveloped in JAVA which allows to design a creature (or a structure) in a user-friendly way Framsticks Experimentation Center is an Internet database ofgenotypes and experiment deregnitionsproposals which can be browseddownloaded and uploaded

The Framstickssystem

167

5 Research experiments51 Comparison of genotype encodingsThere are a number of studies of the evolution of simulated creatures equippedwith realistic physical behavior In such systems the use of a physicalsimulation layer implements a complex genotype-regtness relationshipPhysical interactions between body parts the coupling between control andphysical body and interactions during body development can all add a level ofindirection between the genotype and regtness The complexity of the genotype-regtness relationship offers a potential for rich evolutionary dynamics

The most important element of the genotype-to-regtness relationship is thegenotype-to-phenotype mapping or genotype encoding There is no obvioussimple way to encode a complex phenotype plusmn which consists of a variable-sizestructured body and a matching control system plusmn into a simpler genotypeMoreover an evolutionary algorithm can perform poorly when using a certaingenotype encoding and better when using others for reasons not yetimmediately obvious The employed genotype encoding can have a signiregcanteffect on the performance of the evolution

The Framsticks system has been used as the context of analysis of variousgenotype encodings The performance of the three encodings described inSection 3 was compared in their optimization tasks passive and active heightand velocity maximization (KomosinAcircski and Rotaru-Varga 2001) Thesolutions produced by evolution are considered to be successful for the giventasks in all three cases However there were some important differences in thedegree of success The f 0 encoding performed worse than the two higher-level

Figure 8Two fuzzy rules encodedas a Framsticks neuralnetwork

K3212

168

encodings The most important differences between these encodings are thatf 0 has a minimal bias and is unrestrictive while the higher-level encodings (f1and f4) restrict the search space and introduce a strong bias towardsstructured phenotypes Based on these results we conclude that a morestructured genotype encoding with genetic operators working on a higherlevel is beneregcial in the evolution of 3D agents The presence of a bias towardsstructured phenotypes can overcome the apparent limitation that entire regionsof the search space are not accessible by the search This bias may be useful insome applications (engineering and robotics for example) The signiregcantinmacruence of a chosen encoding can be clearly seen in the obtained agents thosewith f 0 encoding displayed neither order nor structure The two encodingsrestricting morphology to a tree produced more clear constructions and fordevelopmental encoding segmentation and modularity could be observed(Figure 9)

Figure 9Representative agents

for various geneticencodings and height

maximization task

The Framstickssystem

169

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

All these options are useful in various kinds of experiments and performancemeasurements

3 Framework and evolution31 GeneticsThere are multiple genetic encodings (ordfgenotype languagesordm) supported by theFramsticks system each with its own representation and operators Thesystem manipulates and transforms genotype strings in variousrepresentations and ultimately decodes them into the internal representationused by the simulator

Any creature can be completely described using a low-level representationf 0 by listing all of its components and attributes This representation can betreated as a special genotype encoding plusmn special because it is a direct one-to-one mapping Other higher-level encodings convert their representation into thecorresponding f 0 version (possibly through another intermediaryrepresentation) The reverse mapping into higher-level encodings is difregcult

Figure 3Receptors (equilibriumtouch smell) andeffectors (muscles) inFramsticks

Figure 4Flat land smooth slopesand blocks

K3212

160

to compute which is also true for biological phenotype encodings As aconsequence in the general case it is not possible to convert a lower-levelrepresentation into a higher-level one (or a higher-level one into another higher-level one)

Each encoding has its associated genetic operators (mutation crossover andoptional repair) and a decoding procedure which translates a genotype into af 0 representation A new encoding can be added relatively easily byimplementing these components without the need to work with internalrepresentations The Framsticks system is accompanied by the GenotypeDevelopment Kit (GDK) to simplify this process (Ulatowski 2000) Later themost popular encodings are shortly described

The direct low-level or f 0 encoding describes agents exactly as they arerepresented in the simulator This encoding is more of a direct representationthan a proper encoding but it is possible to use it as such It does not use anyhigher-level features to make the genotype more compact or macrexible andbecause of this it is expected that this encoding is not very well suited forevolution Its useful characteristics are that it has a minimal decoding cost andthat it is universal every possible agent can be described using this encodingThese properties make it possible to use the f 0 encoding as an intermediaryrepresentation during the translation from other higher-level encodings

The direct low-level (f 0) genotype consists of a list of descriptions of all theobjects the agent is composed of parts joints neurons and connections Everydescription specireges all the attributes of the object explicitly Generally thisencoding does not impose any restriction on the phenotypes and it even allowsmorphologies with cycles

The recurrent direct encoding (f1) describes all the parts of thecorresponding phenotype Small changes in the genotype cause smallchanges in the resulting creature Body properties are represented locallybut propagate through a creaturersquos structure (with decreasing power) Thatmeans that most of the properties (and neural network connections) aremaintained when a part of a genotype is moved to another place Controlelements (neurons receptors) are associated with the elements under theircontrol (muscles sticks) Only treelike structures can be represented (no cyclesallowed) This encoding is relatively easy for humans to manipulate and designcreatures manually by editing their genotypes

The developmental encoding (f4) is development-oriented similar toencoding applied for evolving neural networks (Gruau et al 1996) Aninteresting merit of developmental encoding is that it can incorporate symmetryand modularity features commonly found in natural systems yet difregcult toformalize f4 is similar to f1 but codes are interpreted as commands by cells(sticks neurons etc) Cells can change their parameters and divide Each cellmaintains its own pointer to the current command in the genetic code Afterdivision cells can execute different codes and thus differentiate themselves

The Framstickssystem

161

The regnal body (phenotype) is the result of a development process it starts withan undifferentiated ancestor cell and ends with a collection of interconnecteddifferentiated cells (sticks neurons and connections)

Each of these encodings has its own genetic operators (mutation andcrossover) Each of them has been carefully designed and tested and eachencoding was based on numerous theoretical considerations More detaileddescription can be found in KomosinAcircski and Rotaru-Varga (2001) Examples ofsimple genotypes and corresponding phenotypes are shown in Figure 5

32 General frameworkThe most important feature of Framsticks is that you may deregne your ownrules for the simulator There are no predetermined laws but a script plusmnexperiment deregnition A script is a set of instructions in some language whichis interpreted by some program and executed

This script deregnes behavior of the Framsticks system in a few associatedareas

Creation of objects in the world The script deregnes where when and howmuch of what objects will be created An object is an evolved organismfood particle or any other element of the world designed by a researcherThus depending on some speciregc script food or obstacles might appearmove and disappear their location might depend on where creatures areetc

Objects interactions Object collisioncontact is an event which maycause some action deregned by the script author For example contact maymean energy ingestion pushing each other destruction or reproduction

Evolution A steady-state (one-at-a-time) selection model where a singlegenotype is inserted into a gene pool at a time can be used But a standard(ie generational replacement) evolutionary algorithm approach is alsopossible (a new gene pool replaces the whole old gene pool) Anotherpossibility is tournament competition for all pairs of genotypes Ingeneral the script can deregne many gene pools and many populations andperform independent evolutions under different conditions

Figure 5Left example of f 1genotypeXXX(XXX(XX))Right example of f4genotype with repetitiongene rr X 5 X RR llX LX LX X

K3212

162

Evaluation criteria are macrexible and do not have to be as simple as theperformances supplied by the simulator (but they will have to be basedsomehow on performances) For example regtness might depend on time orenergy required to fulregll some task or degree of success (distance fromtarget etc)

The script is built of ordfproceduresordm assigned to system events These include thefollowing events

onExpDefLoad plusmn occurs after experiment deregnition was loaded Thisprocedure should prepare the environment create gene pools andpopulations

onExpInit plusmn occurs at the beginning of the experiment

onExpSave plusmn occurs on save experiment request

onExpLoad plusmn occurs on load experiment request After this eventsystem state should resemble the state before onExpSave

onStep plusmn occurs in each simulation step

onBorn plusmn occurs when a new organism is created in the world

onKill plusmn occurs when a creature is removed from the world

on[X]Collision plusmn occurs when an object of group [X] has touchedsome other object

Thus a researcher may deregne the behavior of the whole system byimplementing appropriate actions within these events A single script(experiment deregnition) may use parameters so it usually allows to perform awhole bunch (class) of diversireged experiments

33 Illustrative example (standard experiment deregnition)The regle ordfstandardexpdefordm contains the full source for the script used tooptimize creatures on a steady-state basis with regtness deregned as a weightedsum of their performances (see also Figure 6) This script is quite versatile andcomplex Below its general idea is explained with much simplireged actionsassigned to system eventsonExpDefLoad

create single gene pool ordfGenotypesordm

create two populations ordfCreaturesordm and ordfFoodordm

onExpInit

empty all gene pools and populations

place the beginning genotype in ordfGenotypesordm

The Framstickssystem

163

onStep

if too little food create new object in ordfFoodordm

if too few organisms select a parent from ordfGenotypesordm mutate crossoveror copy it From the resulting genotype create an individual in ordfCreaturesordm

onBorn

move new object into a randomly chosen place in the world

set starting energy according to objectrsquos type

onKill

if ordfCreaturesordm object died save its performance in ordfGenotypesordm (possiblycreating a new genotype) If there are too many genotypes in ordfGenotypesordmremove one

onFoodCollision

send energy portion from ordfFoodordm object to ordfCreatureordm object

4 Additional toolsMany research works concern studies of evolutionary processes theirdynamics and efregciency Various measures and methods have been developedin order to be able to analyze evolution complexity and interaction in theobserved systems Other works try to understand behaviors of artiregcialcreatures regarding them as subjects of survey rather than black boxes withassigned regtness and performance

Artiregcial life systems especially those applied to evolutionary robotics anddesign (Bentley 1999 Funes and Pollack 1998 Lipson and Pollack 2000) arequite complex and it is difregcult to understand the behavior of existing agents in

Figure 6A fragment of softwarewindow with experimentparameters

K3212

164

detail The only way is to observe them carefully and use human intelligence todraw conclusions Usually the behavior of such agents is non-deterministicand their control systems are sophisticated often coupled with morphologyand very strongly connected functionally (Lund et al 1997)

Thus for the purpose of studying behaviors and populations of individualsone needs high-level intelligent support tools (KomosinAcircski and Rotaru-Varga2000) It is not likely that automatic tools will soon be able to produceunderstandable non-trivial explanations of sophisticated artiregcial agentsNonetheless their role and help cannot be ignored Even simple automaticsupport is of great relevance to a human which becomes obvious afterspending hours on investigating relatively simple artiregcial creatures It maybe that in the future some advanced analysis methods developed withinartiregcial life methodology will be useful for real life studies biology andmedicine

One of the main purposes of the Framsticks system is to allow creating andtesting such tools and procedures and to develop methodology needed fortheir use Realistic artiregcial life environments are the right place for suchresearch

41 Clustering of similar individualsSimilarity seems to be a simple property However automatic measures ofsimilarity can help in observation of regularities groups of related individualsetc Similarity can be identireged in many ways including aspects of morphology(body) brain size function behavior performance regtness etc Whateverderegnition is used automatic measure of similarity can be useful for

optimization to introduce artiregcial niches by modiregcation of regtnessvalues (Goldberg 1989)

studies of evolutionary processes and the structure of populations ofindividuals

studies of functionbehavior of agents

reduction of the number of agents to a small subset of interesting diverseunique individuals

inferring dendrograms (and hopefully phylogenetic trees) based ondistances between organisms

For Framsticks we constructed a heuristic method that is able to estimate thedegree of similarity of the two individuals This method treats body as a graph(with material points as vertices and joints as edges) and then tries to matchtwo body structures based on the degrees of vertices as the main piece ofinformation For a more detailed description of this method see KomosinAcircskiet al (2001) and an example is presented in Section 5

The Framstickssystem

165

42 History of evolutionIn real life although we are able to trace genetic relationships within existingcreatures we do not know what happens during mutation and crossing over oftheir genomes Moreover we cannot trace genetic relations in a longer scale andhigher number of individuals

In Framsticks it is possible not only to remember all parent-childrelationships but also to estimate genetic shares of related individuals (howmany genes have mutated or have been exchanged) This allows to draw a realtree of evolution as shown in Figure 7 Vertical axis is time horizontal oneremacrects local degree of genetic dissimilarity (between a pair of individuals)Vertices in the tree are single individuals

43 Fuzzy controlFuzzy control nowadays has become a very popular method of controlling inmany domains of our life washing machines video cameras ABS air-condition etc It is also used for controlling non-linear fast-changing processeswhere quick decisions are more important than exact ones (Yager and Filev1994) Fuzzy control

Figure 7The real tree ofevolution Top singleancestor and beginningof time Black linesrepresent mutationswhite ones arecrossovers

K3212

166

allows for linguistic variables ordfDrive fastordm is much more macrexible thanordfdrive with speed 80-100 kmhordm In fuzzy approach we would say 79 kmhis still fast with degree 95 percent

better comprehension for humans plusmn fuzzy rule if X is Big and Y is Smallthen Z is Medium is easier to understand than crispy one if X is between3222 and 4332 and Y is less than 52 then Z is 192

more macruent ordfnaturalordm change of system states plusmn for example gradualslowing down of a car

For such reasons fuzzy control is developed in Framsticks Therefore evolvedFramsticksrsquo brains may be more understandable for a human Fuzzy controlsimilarly to neural networks can also cope with uncertainty of information plusmnwhen the process is compound depends on lottery events or the measurementis burden with error Fuzzy approach can manage this inconvenience bygeneralization of information To incorporate this approach in Framsticksneural network model three additional types of neurons are needed

difference neuron plusmn to compare current value of the sensor to the previousone

rule neuron plusmn to represent a fuzzy rule

aggregate neuron plusmn to gather answers of the rule neurons and calculatecrisp output

The two sample fuzzy rules are

IF gyroscope is big+ AND Dgyroscope is small+

THEN bend muscle medium 2

IF gyroscope is medium 2 AND Dgyroscope is zero+

THEN bend muscle big+

where gyroscope is the measurement taken from the gyroscope receptorDgyroscope is the change in its value and linguistic terms ordfbig+ordm etccorrespond to fuzzy intervals of values The representation of these two rules inthe Framsticks neural network is shown in Figure 8

44 Helper softwareTogether with continuous development of extensions to the Framsticks systemitself there are some external tools created Fred is a visual Framsticks Editordeveloped in JAVA which allows to design a creature (or a structure) in a user-friendly way Framsticks Experimentation Center is an Internet database ofgenotypes and experiment deregnitionsproposals which can be browseddownloaded and uploaded

The Framstickssystem

167

5 Research experiments51 Comparison of genotype encodingsThere are a number of studies of the evolution of simulated creatures equippedwith realistic physical behavior In such systems the use of a physicalsimulation layer implements a complex genotype-regtness relationshipPhysical interactions between body parts the coupling between control andphysical body and interactions during body development can all add a level ofindirection between the genotype and regtness The complexity of the genotype-regtness relationship offers a potential for rich evolutionary dynamics

The most important element of the genotype-to-regtness relationship is thegenotype-to-phenotype mapping or genotype encoding There is no obvioussimple way to encode a complex phenotype plusmn which consists of a variable-sizestructured body and a matching control system plusmn into a simpler genotypeMoreover an evolutionary algorithm can perform poorly when using a certaingenotype encoding and better when using others for reasons not yetimmediately obvious The employed genotype encoding can have a signiregcanteffect on the performance of the evolution

The Framsticks system has been used as the context of analysis of variousgenotype encodings The performance of the three encodings described inSection 3 was compared in their optimization tasks passive and active heightand velocity maximization (KomosinAcircski and Rotaru-Varga 2001) Thesolutions produced by evolution are considered to be successful for the giventasks in all three cases However there were some important differences in thedegree of success The f 0 encoding performed worse than the two higher-level

Figure 8Two fuzzy rules encodedas a Framsticks neuralnetwork

K3212

168

encodings The most important differences between these encodings are thatf 0 has a minimal bias and is unrestrictive while the higher-level encodings (f1and f4) restrict the search space and introduce a strong bias towardsstructured phenotypes Based on these results we conclude that a morestructured genotype encoding with genetic operators working on a higherlevel is beneregcial in the evolution of 3D agents The presence of a bias towardsstructured phenotypes can overcome the apparent limitation that entire regionsof the search space are not accessible by the search This bias may be useful insome applications (engineering and robotics for example) The signiregcantinmacruence of a chosen encoding can be clearly seen in the obtained agents thosewith f 0 encoding displayed neither order nor structure The two encodingsrestricting morphology to a tree produced more clear constructions and fordevelopmental encoding segmentation and modularity could be observed(Figure 9)

Figure 9Representative agents

for various geneticencodings and height

maximization task

The Framstickssystem

169

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

to compute which is also true for biological phenotype encodings As aconsequence in the general case it is not possible to convert a lower-levelrepresentation into a higher-level one (or a higher-level one into another higher-level one)

Each encoding has its associated genetic operators (mutation crossover andoptional repair) and a decoding procedure which translates a genotype into af 0 representation A new encoding can be added relatively easily byimplementing these components without the need to work with internalrepresentations The Framsticks system is accompanied by the GenotypeDevelopment Kit (GDK) to simplify this process (Ulatowski 2000) Later themost popular encodings are shortly described

The direct low-level or f 0 encoding describes agents exactly as they arerepresented in the simulator This encoding is more of a direct representationthan a proper encoding but it is possible to use it as such It does not use anyhigher-level features to make the genotype more compact or macrexible andbecause of this it is expected that this encoding is not very well suited forevolution Its useful characteristics are that it has a minimal decoding cost andthat it is universal every possible agent can be described using this encodingThese properties make it possible to use the f 0 encoding as an intermediaryrepresentation during the translation from other higher-level encodings

The direct low-level (f 0) genotype consists of a list of descriptions of all theobjects the agent is composed of parts joints neurons and connections Everydescription specireges all the attributes of the object explicitly Generally thisencoding does not impose any restriction on the phenotypes and it even allowsmorphologies with cycles

The recurrent direct encoding (f1) describes all the parts of thecorresponding phenotype Small changes in the genotype cause smallchanges in the resulting creature Body properties are represented locallybut propagate through a creaturersquos structure (with decreasing power) Thatmeans that most of the properties (and neural network connections) aremaintained when a part of a genotype is moved to another place Controlelements (neurons receptors) are associated with the elements under theircontrol (muscles sticks) Only treelike structures can be represented (no cyclesallowed) This encoding is relatively easy for humans to manipulate and designcreatures manually by editing their genotypes

The developmental encoding (f4) is development-oriented similar toencoding applied for evolving neural networks (Gruau et al 1996) Aninteresting merit of developmental encoding is that it can incorporate symmetryand modularity features commonly found in natural systems yet difregcult toformalize f4 is similar to f1 but codes are interpreted as commands by cells(sticks neurons etc) Cells can change their parameters and divide Each cellmaintains its own pointer to the current command in the genetic code Afterdivision cells can execute different codes and thus differentiate themselves

The Framstickssystem

161

The regnal body (phenotype) is the result of a development process it starts withan undifferentiated ancestor cell and ends with a collection of interconnecteddifferentiated cells (sticks neurons and connections)

Each of these encodings has its own genetic operators (mutation andcrossover) Each of them has been carefully designed and tested and eachencoding was based on numerous theoretical considerations More detaileddescription can be found in KomosinAcircski and Rotaru-Varga (2001) Examples ofsimple genotypes and corresponding phenotypes are shown in Figure 5

32 General frameworkThe most important feature of Framsticks is that you may deregne your ownrules for the simulator There are no predetermined laws but a script plusmnexperiment deregnition A script is a set of instructions in some language whichis interpreted by some program and executed

This script deregnes behavior of the Framsticks system in a few associatedareas

Creation of objects in the world The script deregnes where when and howmuch of what objects will be created An object is an evolved organismfood particle or any other element of the world designed by a researcherThus depending on some speciregc script food or obstacles might appearmove and disappear their location might depend on where creatures areetc

Objects interactions Object collisioncontact is an event which maycause some action deregned by the script author For example contact maymean energy ingestion pushing each other destruction or reproduction

Evolution A steady-state (one-at-a-time) selection model where a singlegenotype is inserted into a gene pool at a time can be used But a standard(ie generational replacement) evolutionary algorithm approach is alsopossible (a new gene pool replaces the whole old gene pool) Anotherpossibility is tournament competition for all pairs of genotypes Ingeneral the script can deregne many gene pools and many populations andperform independent evolutions under different conditions

Figure 5Left example of f 1genotypeXXX(XXX(XX))Right example of f4genotype with repetitiongene rr X 5 X RR llX LX LX X

K3212

162

Evaluation criteria are macrexible and do not have to be as simple as theperformances supplied by the simulator (but they will have to be basedsomehow on performances) For example regtness might depend on time orenergy required to fulregll some task or degree of success (distance fromtarget etc)

The script is built of ordfproceduresordm assigned to system events These include thefollowing events

onExpDefLoad plusmn occurs after experiment deregnition was loaded Thisprocedure should prepare the environment create gene pools andpopulations

onExpInit plusmn occurs at the beginning of the experiment

onExpSave plusmn occurs on save experiment request

onExpLoad plusmn occurs on load experiment request After this eventsystem state should resemble the state before onExpSave

onStep plusmn occurs in each simulation step

onBorn plusmn occurs when a new organism is created in the world

onKill plusmn occurs when a creature is removed from the world

on[X]Collision plusmn occurs when an object of group [X] has touchedsome other object

Thus a researcher may deregne the behavior of the whole system byimplementing appropriate actions within these events A single script(experiment deregnition) may use parameters so it usually allows to perform awhole bunch (class) of diversireged experiments

33 Illustrative example (standard experiment deregnition)The regle ordfstandardexpdefordm contains the full source for the script used tooptimize creatures on a steady-state basis with regtness deregned as a weightedsum of their performances (see also Figure 6) This script is quite versatile andcomplex Below its general idea is explained with much simplireged actionsassigned to system eventsonExpDefLoad

create single gene pool ordfGenotypesordm

create two populations ordfCreaturesordm and ordfFoodordm

onExpInit

empty all gene pools and populations

place the beginning genotype in ordfGenotypesordm

The Framstickssystem

163

onStep

if too little food create new object in ordfFoodordm

if too few organisms select a parent from ordfGenotypesordm mutate crossoveror copy it From the resulting genotype create an individual in ordfCreaturesordm

onBorn

move new object into a randomly chosen place in the world

set starting energy according to objectrsquos type

onKill

if ordfCreaturesordm object died save its performance in ordfGenotypesordm (possiblycreating a new genotype) If there are too many genotypes in ordfGenotypesordmremove one

onFoodCollision

send energy portion from ordfFoodordm object to ordfCreatureordm object

4 Additional toolsMany research works concern studies of evolutionary processes theirdynamics and efregciency Various measures and methods have been developedin order to be able to analyze evolution complexity and interaction in theobserved systems Other works try to understand behaviors of artiregcialcreatures regarding them as subjects of survey rather than black boxes withassigned regtness and performance

Artiregcial life systems especially those applied to evolutionary robotics anddesign (Bentley 1999 Funes and Pollack 1998 Lipson and Pollack 2000) arequite complex and it is difregcult to understand the behavior of existing agents in

Figure 6A fragment of softwarewindow with experimentparameters

K3212

164

detail The only way is to observe them carefully and use human intelligence todraw conclusions Usually the behavior of such agents is non-deterministicand their control systems are sophisticated often coupled with morphologyand very strongly connected functionally (Lund et al 1997)

Thus for the purpose of studying behaviors and populations of individualsone needs high-level intelligent support tools (KomosinAcircski and Rotaru-Varga2000) It is not likely that automatic tools will soon be able to produceunderstandable non-trivial explanations of sophisticated artiregcial agentsNonetheless their role and help cannot be ignored Even simple automaticsupport is of great relevance to a human which becomes obvious afterspending hours on investigating relatively simple artiregcial creatures It maybe that in the future some advanced analysis methods developed withinartiregcial life methodology will be useful for real life studies biology andmedicine

One of the main purposes of the Framsticks system is to allow creating andtesting such tools and procedures and to develop methodology needed fortheir use Realistic artiregcial life environments are the right place for suchresearch

41 Clustering of similar individualsSimilarity seems to be a simple property However automatic measures ofsimilarity can help in observation of regularities groups of related individualsetc Similarity can be identireged in many ways including aspects of morphology(body) brain size function behavior performance regtness etc Whateverderegnition is used automatic measure of similarity can be useful for

optimization to introduce artiregcial niches by modiregcation of regtnessvalues (Goldberg 1989)

studies of evolutionary processes and the structure of populations ofindividuals

studies of functionbehavior of agents

reduction of the number of agents to a small subset of interesting diverseunique individuals

inferring dendrograms (and hopefully phylogenetic trees) based ondistances between organisms

For Framsticks we constructed a heuristic method that is able to estimate thedegree of similarity of the two individuals This method treats body as a graph(with material points as vertices and joints as edges) and then tries to matchtwo body structures based on the degrees of vertices as the main piece ofinformation For a more detailed description of this method see KomosinAcircskiet al (2001) and an example is presented in Section 5

The Framstickssystem

165

42 History of evolutionIn real life although we are able to trace genetic relationships within existingcreatures we do not know what happens during mutation and crossing over oftheir genomes Moreover we cannot trace genetic relations in a longer scale andhigher number of individuals

In Framsticks it is possible not only to remember all parent-childrelationships but also to estimate genetic shares of related individuals (howmany genes have mutated or have been exchanged) This allows to draw a realtree of evolution as shown in Figure 7 Vertical axis is time horizontal oneremacrects local degree of genetic dissimilarity (between a pair of individuals)Vertices in the tree are single individuals

43 Fuzzy controlFuzzy control nowadays has become a very popular method of controlling inmany domains of our life washing machines video cameras ABS air-condition etc It is also used for controlling non-linear fast-changing processeswhere quick decisions are more important than exact ones (Yager and Filev1994) Fuzzy control

Figure 7The real tree ofevolution Top singleancestor and beginningof time Black linesrepresent mutationswhite ones arecrossovers

K3212

166

allows for linguistic variables ordfDrive fastordm is much more macrexible thanordfdrive with speed 80-100 kmhordm In fuzzy approach we would say 79 kmhis still fast with degree 95 percent

better comprehension for humans plusmn fuzzy rule if X is Big and Y is Smallthen Z is Medium is easier to understand than crispy one if X is between3222 and 4332 and Y is less than 52 then Z is 192

more macruent ordfnaturalordm change of system states plusmn for example gradualslowing down of a car

For such reasons fuzzy control is developed in Framsticks Therefore evolvedFramsticksrsquo brains may be more understandable for a human Fuzzy controlsimilarly to neural networks can also cope with uncertainty of information plusmnwhen the process is compound depends on lottery events or the measurementis burden with error Fuzzy approach can manage this inconvenience bygeneralization of information To incorporate this approach in Framsticksneural network model three additional types of neurons are needed

difference neuron plusmn to compare current value of the sensor to the previousone

rule neuron plusmn to represent a fuzzy rule

aggregate neuron plusmn to gather answers of the rule neurons and calculatecrisp output

The two sample fuzzy rules are

IF gyroscope is big+ AND Dgyroscope is small+

THEN bend muscle medium 2

IF gyroscope is medium 2 AND Dgyroscope is zero+

THEN bend muscle big+

where gyroscope is the measurement taken from the gyroscope receptorDgyroscope is the change in its value and linguistic terms ordfbig+ordm etccorrespond to fuzzy intervals of values The representation of these two rules inthe Framsticks neural network is shown in Figure 8

44 Helper softwareTogether with continuous development of extensions to the Framsticks systemitself there are some external tools created Fred is a visual Framsticks Editordeveloped in JAVA which allows to design a creature (or a structure) in a user-friendly way Framsticks Experimentation Center is an Internet database ofgenotypes and experiment deregnitionsproposals which can be browseddownloaded and uploaded

The Framstickssystem

167

5 Research experiments51 Comparison of genotype encodingsThere are a number of studies of the evolution of simulated creatures equippedwith realistic physical behavior In such systems the use of a physicalsimulation layer implements a complex genotype-regtness relationshipPhysical interactions between body parts the coupling between control andphysical body and interactions during body development can all add a level ofindirection between the genotype and regtness The complexity of the genotype-regtness relationship offers a potential for rich evolutionary dynamics

The most important element of the genotype-to-regtness relationship is thegenotype-to-phenotype mapping or genotype encoding There is no obvioussimple way to encode a complex phenotype plusmn which consists of a variable-sizestructured body and a matching control system plusmn into a simpler genotypeMoreover an evolutionary algorithm can perform poorly when using a certaingenotype encoding and better when using others for reasons not yetimmediately obvious The employed genotype encoding can have a signiregcanteffect on the performance of the evolution

The Framsticks system has been used as the context of analysis of variousgenotype encodings The performance of the three encodings described inSection 3 was compared in their optimization tasks passive and active heightand velocity maximization (KomosinAcircski and Rotaru-Varga 2001) Thesolutions produced by evolution are considered to be successful for the giventasks in all three cases However there were some important differences in thedegree of success The f 0 encoding performed worse than the two higher-level

Figure 8Two fuzzy rules encodedas a Framsticks neuralnetwork

K3212

168

encodings The most important differences between these encodings are thatf 0 has a minimal bias and is unrestrictive while the higher-level encodings (f1and f4) restrict the search space and introduce a strong bias towardsstructured phenotypes Based on these results we conclude that a morestructured genotype encoding with genetic operators working on a higherlevel is beneregcial in the evolution of 3D agents The presence of a bias towardsstructured phenotypes can overcome the apparent limitation that entire regionsof the search space are not accessible by the search This bias may be useful insome applications (engineering and robotics for example) The signiregcantinmacruence of a chosen encoding can be clearly seen in the obtained agents thosewith f 0 encoding displayed neither order nor structure The two encodingsrestricting morphology to a tree produced more clear constructions and fordevelopmental encoding segmentation and modularity could be observed(Figure 9)

Figure 9Representative agents

for various geneticencodings and height

maximization task

The Framstickssystem

169

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

The regnal body (phenotype) is the result of a development process it starts withan undifferentiated ancestor cell and ends with a collection of interconnecteddifferentiated cells (sticks neurons and connections)

Each of these encodings has its own genetic operators (mutation andcrossover) Each of them has been carefully designed and tested and eachencoding was based on numerous theoretical considerations More detaileddescription can be found in KomosinAcircski and Rotaru-Varga (2001) Examples ofsimple genotypes and corresponding phenotypes are shown in Figure 5

32 General frameworkThe most important feature of Framsticks is that you may deregne your ownrules for the simulator There are no predetermined laws but a script plusmnexperiment deregnition A script is a set of instructions in some language whichis interpreted by some program and executed

This script deregnes behavior of the Framsticks system in a few associatedareas

Creation of objects in the world The script deregnes where when and howmuch of what objects will be created An object is an evolved organismfood particle or any other element of the world designed by a researcherThus depending on some speciregc script food or obstacles might appearmove and disappear their location might depend on where creatures areetc

Objects interactions Object collisioncontact is an event which maycause some action deregned by the script author For example contact maymean energy ingestion pushing each other destruction or reproduction

Evolution A steady-state (one-at-a-time) selection model where a singlegenotype is inserted into a gene pool at a time can be used But a standard(ie generational replacement) evolutionary algorithm approach is alsopossible (a new gene pool replaces the whole old gene pool) Anotherpossibility is tournament competition for all pairs of genotypes Ingeneral the script can deregne many gene pools and many populations andperform independent evolutions under different conditions

Figure 5Left example of f 1genotypeXXX(XXX(XX))Right example of f4genotype with repetitiongene rr X 5 X RR llX LX LX X

K3212

162

Evaluation criteria are macrexible and do not have to be as simple as theperformances supplied by the simulator (but they will have to be basedsomehow on performances) For example regtness might depend on time orenergy required to fulregll some task or degree of success (distance fromtarget etc)

The script is built of ordfproceduresordm assigned to system events These include thefollowing events

onExpDefLoad plusmn occurs after experiment deregnition was loaded Thisprocedure should prepare the environment create gene pools andpopulations

onExpInit plusmn occurs at the beginning of the experiment

onExpSave plusmn occurs on save experiment request

onExpLoad plusmn occurs on load experiment request After this eventsystem state should resemble the state before onExpSave

onStep plusmn occurs in each simulation step

onBorn plusmn occurs when a new organism is created in the world

onKill plusmn occurs when a creature is removed from the world

on[X]Collision plusmn occurs when an object of group [X] has touchedsome other object

Thus a researcher may deregne the behavior of the whole system byimplementing appropriate actions within these events A single script(experiment deregnition) may use parameters so it usually allows to perform awhole bunch (class) of diversireged experiments

33 Illustrative example (standard experiment deregnition)The regle ordfstandardexpdefordm contains the full source for the script used tooptimize creatures on a steady-state basis with regtness deregned as a weightedsum of their performances (see also Figure 6) This script is quite versatile andcomplex Below its general idea is explained with much simplireged actionsassigned to system eventsonExpDefLoad

create single gene pool ordfGenotypesordm

create two populations ordfCreaturesordm and ordfFoodordm

onExpInit

empty all gene pools and populations

place the beginning genotype in ordfGenotypesordm

The Framstickssystem

163

onStep

if too little food create new object in ordfFoodordm

if too few organisms select a parent from ordfGenotypesordm mutate crossoveror copy it From the resulting genotype create an individual in ordfCreaturesordm

onBorn

move new object into a randomly chosen place in the world

set starting energy according to objectrsquos type

onKill

if ordfCreaturesordm object died save its performance in ordfGenotypesordm (possiblycreating a new genotype) If there are too many genotypes in ordfGenotypesordmremove one

onFoodCollision

send energy portion from ordfFoodordm object to ordfCreatureordm object

4 Additional toolsMany research works concern studies of evolutionary processes theirdynamics and efregciency Various measures and methods have been developedin order to be able to analyze evolution complexity and interaction in theobserved systems Other works try to understand behaviors of artiregcialcreatures regarding them as subjects of survey rather than black boxes withassigned regtness and performance

Artiregcial life systems especially those applied to evolutionary robotics anddesign (Bentley 1999 Funes and Pollack 1998 Lipson and Pollack 2000) arequite complex and it is difregcult to understand the behavior of existing agents in

Figure 6A fragment of softwarewindow with experimentparameters

K3212

164

detail The only way is to observe them carefully and use human intelligence todraw conclusions Usually the behavior of such agents is non-deterministicand their control systems are sophisticated often coupled with morphologyand very strongly connected functionally (Lund et al 1997)

Thus for the purpose of studying behaviors and populations of individualsone needs high-level intelligent support tools (KomosinAcircski and Rotaru-Varga2000) It is not likely that automatic tools will soon be able to produceunderstandable non-trivial explanations of sophisticated artiregcial agentsNonetheless their role and help cannot be ignored Even simple automaticsupport is of great relevance to a human which becomes obvious afterspending hours on investigating relatively simple artiregcial creatures It maybe that in the future some advanced analysis methods developed withinartiregcial life methodology will be useful for real life studies biology andmedicine

One of the main purposes of the Framsticks system is to allow creating andtesting such tools and procedures and to develop methodology needed fortheir use Realistic artiregcial life environments are the right place for suchresearch

41 Clustering of similar individualsSimilarity seems to be a simple property However automatic measures ofsimilarity can help in observation of regularities groups of related individualsetc Similarity can be identireged in many ways including aspects of morphology(body) brain size function behavior performance regtness etc Whateverderegnition is used automatic measure of similarity can be useful for

optimization to introduce artiregcial niches by modiregcation of regtnessvalues (Goldberg 1989)

studies of evolutionary processes and the structure of populations ofindividuals

studies of functionbehavior of agents

reduction of the number of agents to a small subset of interesting diverseunique individuals

inferring dendrograms (and hopefully phylogenetic trees) based ondistances between organisms

For Framsticks we constructed a heuristic method that is able to estimate thedegree of similarity of the two individuals This method treats body as a graph(with material points as vertices and joints as edges) and then tries to matchtwo body structures based on the degrees of vertices as the main piece ofinformation For a more detailed description of this method see KomosinAcircskiet al (2001) and an example is presented in Section 5

The Framstickssystem

165

42 History of evolutionIn real life although we are able to trace genetic relationships within existingcreatures we do not know what happens during mutation and crossing over oftheir genomes Moreover we cannot trace genetic relations in a longer scale andhigher number of individuals

In Framsticks it is possible not only to remember all parent-childrelationships but also to estimate genetic shares of related individuals (howmany genes have mutated or have been exchanged) This allows to draw a realtree of evolution as shown in Figure 7 Vertical axis is time horizontal oneremacrects local degree of genetic dissimilarity (between a pair of individuals)Vertices in the tree are single individuals

43 Fuzzy controlFuzzy control nowadays has become a very popular method of controlling inmany domains of our life washing machines video cameras ABS air-condition etc It is also used for controlling non-linear fast-changing processeswhere quick decisions are more important than exact ones (Yager and Filev1994) Fuzzy control

Figure 7The real tree ofevolution Top singleancestor and beginningof time Black linesrepresent mutationswhite ones arecrossovers

K3212

166

allows for linguistic variables ordfDrive fastordm is much more macrexible thanordfdrive with speed 80-100 kmhordm In fuzzy approach we would say 79 kmhis still fast with degree 95 percent

better comprehension for humans plusmn fuzzy rule if X is Big and Y is Smallthen Z is Medium is easier to understand than crispy one if X is between3222 and 4332 and Y is less than 52 then Z is 192

more macruent ordfnaturalordm change of system states plusmn for example gradualslowing down of a car

For such reasons fuzzy control is developed in Framsticks Therefore evolvedFramsticksrsquo brains may be more understandable for a human Fuzzy controlsimilarly to neural networks can also cope with uncertainty of information plusmnwhen the process is compound depends on lottery events or the measurementis burden with error Fuzzy approach can manage this inconvenience bygeneralization of information To incorporate this approach in Framsticksneural network model three additional types of neurons are needed

difference neuron plusmn to compare current value of the sensor to the previousone

rule neuron plusmn to represent a fuzzy rule

aggregate neuron plusmn to gather answers of the rule neurons and calculatecrisp output

The two sample fuzzy rules are

IF gyroscope is big+ AND Dgyroscope is small+

THEN bend muscle medium 2

IF gyroscope is medium 2 AND Dgyroscope is zero+

THEN bend muscle big+

where gyroscope is the measurement taken from the gyroscope receptorDgyroscope is the change in its value and linguistic terms ordfbig+ordm etccorrespond to fuzzy intervals of values The representation of these two rules inthe Framsticks neural network is shown in Figure 8

44 Helper softwareTogether with continuous development of extensions to the Framsticks systemitself there are some external tools created Fred is a visual Framsticks Editordeveloped in JAVA which allows to design a creature (or a structure) in a user-friendly way Framsticks Experimentation Center is an Internet database ofgenotypes and experiment deregnitionsproposals which can be browseddownloaded and uploaded

The Framstickssystem

167

5 Research experiments51 Comparison of genotype encodingsThere are a number of studies of the evolution of simulated creatures equippedwith realistic physical behavior In such systems the use of a physicalsimulation layer implements a complex genotype-regtness relationshipPhysical interactions between body parts the coupling between control andphysical body and interactions during body development can all add a level ofindirection between the genotype and regtness The complexity of the genotype-regtness relationship offers a potential for rich evolutionary dynamics

The most important element of the genotype-to-regtness relationship is thegenotype-to-phenotype mapping or genotype encoding There is no obvioussimple way to encode a complex phenotype plusmn which consists of a variable-sizestructured body and a matching control system plusmn into a simpler genotypeMoreover an evolutionary algorithm can perform poorly when using a certaingenotype encoding and better when using others for reasons not yetimmediately obvious The employed genotype encoding can have a signiregcanteffect on the performance of the evolution

The Framsticks system has been used as the context of analysis of variousgenotype encodings The performance of the three encodings described inSection 3 was compared in their optimization tasks passive and active heightand velocity maximization (KomosinAcircski and Rotaru-Varga 2001) Thesolutions produced by evolution are considered to be successful for the giventasks in all three cases However there were some important differences in thedegree of success The f 0 encoding performed worse than the two higher-level

Figure 8Two fuzzy rules encodedas a Framsticks neuralnetwork

K3212

168

encodings The most important differences between these encodings are thatf 0 has a minimal bias and is unrestrictive while the higher-level encodings (f1and f4) restrict the search space and introduce a strong bias towardsstructured phenotypes Based on these results we conclude that a morestructured genotype encoding with genetic operators working on a higherlevel is beneregcial in the evolution of 3D agents The presence of a bias towardsstructured phenotypes can overcome the apparent limitation that entire regionsof the search space are not accessible by the search This bias may be useful insome applications (engineering and robotics for example) The signiregcantinmacruence of a chosen encoding can be clearly seen in the obtained agents thosewith f 0 encoding displayed neither order nor structure The two encodingsrestricting morphology to a tree produced more clear constructions and fordevelopmental encoding segmentation and modularity could be observed(Figure 9)

Figure 9Representative agents

for various geneticencodings and height

maximization task

The Framstickssystem

169

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

Evaluation criteria are macrexible and do not have to be as simple as theperformances supplied by the simulator (but they will have to be basedsomehow on performances) For example regtness might depend on time orenergy required to fulregll some task or degree of success (distance fromtarget etc)

The script is built of ordfproceduresordm assigned to system events These include thefollowing events

onExpDefLoad plusmn occurs after experiment deregnition was loaded Thisprocedure should prepare the environment create gene pools andpopulations

onExpInit plusmn occurs at the beginning of the experiment

onExpSave plusmn occurs on save experiment request

onExpLoad plusmn occurs on load experiment request After this eventsystem state should resemble the state before onExpSave

onStep plusmn occurs in each simulation step

onBorn plusmn occurs when a new organism is created in the world

onKill plusmn occurs when a creature is removed from the world

on[X]Collision plusmn occurs when an object of group [X] has touchedsome other object

Thus a researcher may deregne the behavior of the whole system byimplementing appropriate actions within these events A single script(experiment deregnition) may use parameters so it usually allows to perform awhole bunch (class) of diversireged experiments

33 Illustrative example (standard experiment deregnition)The regle ordfstandardexpdefordm contains the full source for the script used tooptimize creatures on a steady-state basis with regtness deregned as a weightedsum of their performances (see also Figure 6) This script is quite versatile andcomplex Below its general idea is explained with much simplireged actionsassigned to system eventsonExpDefLoad

create single gene pool ordfGenotypesordm

create two populations ordfCreaturesordm and ordfFoodordm

onExpInit

empty all gene pools and populations

place the beginning genotype in ordfGenotypesordm

The Framstickssystem

163

onStep

if too little food create new object in ordfFoodordm

if too few organisms select a parent from ordfGenotypesordm mutate crossoveror copy it From the resulting genotype create an individual in ordfCreaturesordm

onBorn

move new object into a randomly chosen place in the world

set starting energy according to objectrsquos type

onKill

if ordfCreaturesordm object died save its performance in ordfGenotypesordm (possiblycreating a new genotype) If there are too many genotypes in ordfGenotypesordmremove one

onFoodCollision

send energy portion from ordfFoodordm object to ordfCreatureordm object

4 Additional toolsMany research works concern studies of evolutionary processes theirdynamics and efregciency Various measures and methods have been developedin order to be able to analyze evolution complexity and interaction in theobserved systems Other works try to understand behaviors of artiregcialcreatures regarding them as subjects of survey rather than black boxes withassigned regtness and performance

Artiregcial life systems especially those applied to evolutionary robotics anddesign (Bentley 1999 Funes and Pollack 1998 Lipson and Pollack 2000) arequite complex and it is difregcult to understand the behavior of existing agents in

Figure 6A fragment of softwarewindow with experimentparameters

K3212

164

detail The only way is to observe them carefully and use human intelligence todraw conclusions Usually the behavior of such agents is non-deterministicand their control systems are sophisticated often coupled with morphologyand very strongly connected functionally (Lund et al 1997)

Thus for the purpose of studying behaviors and populations of individualsone needs high-level intelligent support tools (KomosinAcircski and Rotaru-Varga2000) It is not likely that automatic tools will soon be able to produceunderstandable non-trivial explanations of sophisticated artiregcial agentsNonetheless their role and help cannot be ignored Even simple automaticsupport is of great relevance to a human which becomes obvious afterspending hours on investigating relatively simple artiregcial creatures It maybe that in the future some advanced analysis methods developed withinartiregcial life methodology will be useful for real life studies biology andmedicine

One of the main purposes of the Framsticks system is to allow creating andtesting such tools and procedures and to develop methodology needed fortheir use Realistic artiregcial life environments are the right place for suchresearch

41 Clustering of similar individualsSimilarity seems to be a simple property However automatic measures ofsimilarity can help in observation of regularities groups of related individualsetc Similarity can be identireged in many ways including aspects of morphology(body) brain size function behavior performance regtness etc Whateverderegnition is used automatic measure of similarity can be useful for

optimization to introduce artiregcial niches by modiregcation of regtnessvalues (Goldberg 1989)

studies of evolutionary processes and the structure of populations ofindividuals

studies of functionbehavior of agents

reduction of the number of agents to a small subset of interesting diverseunique individuals

inferring dendrograms (and hopefully phylogenetic trees) based ondistances between organisms

For Framsticks we constructed a heuristic method that is able to estimate thedegree of similarity of the two individuals This method treats body as a graph(with material points as vertices and joints as edges) and then tries to matchtwo body structures based on the degrees of vertices as the main piece ofinformation For a more detailed description of this method see KomosinAcircskiet al (2001) and an example is presented in Section 5

The Framstickssystem

165

42 History of evolutionIn real life although we are able to trace genetic relationships within existingcreatures we do not know what happens during mutation and crossing over oftheir genomes Moreover we cannot trace genetic relations in a longer scale andhigher number of individuals

In Framsticks it is possible not only to remember all parent-childrelationships but also to estimate genetic shares of related individuals (howmany genes have mutated or have been exchanged) This allows to draw a realtree of evolution as shown in Figure 7 Vertical axis is time horizontal oneremacrects local degree of genetic dissimilarity (between a pair of individuals)Vertices in the tree are single individuals

43 Fuzzy controlFuzzy control nowadays has become a very popular method of controlling inmany domains of our life washing machines video cameras ABS air-condition etc It is also used for controlling non-linear fast-changing processeswhere quick decisions are more important than exact ones (Yager and Filev1994) Fuzzy control

Figure 7The real tree ofevolution Top singleancestor and beginningof time Black linesrepresent mutationswhite ones arecrossovers

K3212

166

allows for linguistic variables ordfDrive fastordm is much more macrexible thanordfdrive with speed 80-100 kmhordm In fuzzy approach we would say 79 kmhis still fast with degree 95 percent

better comprehension for humans plusmn fuzzy rule if X is Big and Y is Smallthen Z is Medium is easier to understand than crispy one if X is between3222 and 4332 and Y is less than 52 then Z is 192

more macruent ordfnaturalordm change of system states plusmn for example gradualslowing down of a car

For such reasons fuzzy control is developed in Framsticks Therefore evolvedFramsticksrsquo brains may be more understandable for a human Fuzzy controlsimilarly to neural networks can also cope with uncertainty of information plusmnwhen the process is compound depends on lottery events or the measurementis burden with error Fuzzy approach can manage this inconvenience bygeneralization of information To incorporate this approach in Framsticksneural network model three additional types of neurons are needed

difference neuron plusmn to compare current value of the sensor to the previousone

rule neuron plusmn to represent a fuzzy rule

aggregate neuron plusmn to gather answers of the rule neurons and calculatecrisp output

The two sample fuzzy rules are

IF gyroscope is big+ AND Dgyroscope is small+

THEN bend muscle medium 2

IF gyroscope is medium 2 AND Dgyroscope is zero+

THEN bend muscle big+

where gyroscope is the measurement taken from the gyroscope receptorDgyroscope is the change in its value and linguistic terms ordfbig+ordm etccorrespond to fuzzy intervals of values The representation of these two rules inthe Framsticks neural network is shown in Figure 8

44 Helper softwareTogether with continuous development of extensions to the Framsticks systemitself there are some external tools created Fred is a visual Framsticks Editordeveloped in JAVA which allows to design a creature (or a structure) in a user-friendly way Framsticks Experimentation Center is an Internet database ofgenotypes and experiment deregnitionsproposals which can be browseddownloaded and uploaded

The Framstickssystem

167

5 Research experiments51 Comparison of genotype encodingsThere are a number of studies of the evolution of simulated creatures equippedwith realistic physical behavior In such systems the use of a physicalsimulation layer implements a complex genotype-regtness relationshipPhysical interactions between body parts the coupling between control andphysical body and interactions during body development can all add a level ofindirection between the genotype and regtness The complexity of the genotype-regtness relationship offers a potential for rich evolutionary dynamics

The most important element of the genotype-to-regtness relationship is thegenotype-to-phenotype mapping or genotype encoding There is no obvioussimple way to encode a complex phenotype plusmn which consists of a variable-sizestructured body and a matching control system plusmn into a simpler genotypeMoreover an evolutionary algorithm can perform poorly when using a certaingenotype encoding and better when using others for reasons not yetimmediately obvious The employed genotype encoding can have a signiregcanteffect on the performance of the evolution

The Framsticks system has been used as the context of analysis of variousgenotype encodings The performance of the three encodings described inSection 3 was compared in their optimization tasks passive and active heightand velocity maximization (KomosinAcircski and Rotaru-Varga 2001) Thesolutions produced by evolution are considered to be successful for the giventasks in all three cases However there were some important differences in thedegree of success The f 0 encoding performed worse than the two higher-level

Figure 8Two fuzzy rules encodedas a Framsticks neuralnetwork

K3212

168

encodings The most important differences between these encodings are thatf 0 has a minimal bias and is unrestrictive while the higher-level encodings (f1and f4) restrict the search space and introduce a strong bias towardsstructured phenotypes Based on these results we conclude that a morestructured genotype encoding with genetic operators working on a higherlevel is beneregcial in the evolution of 3D agents The presence of a bias towardsstructured phenotypes can overcome the apparent limitation that entire regionsof the search space are not accessible by the search This bias may be useful insome applications (engineering and robotics for example) The signiregcantinmacruence of a chosen encoding can be clearly seen in the obtained agents thosewith f 0 encoding displayed neither order nor structure The two encodingsrestricting morphology to a tree produced more clear constructions and fordevelopmental encoding segmentation and modularity could be observed(Figure 9)

Figure 9Representative agents

for various geneticencodings and height

maximization task

The Framstickssystem

169

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

onStep

if too little food create new object in ordfFoodordm

if too few organisms select a parent from ordfGenotypesordm mutate crossoveror copy it From the resulting genotype create an individual in ordfCreaturesordm

onBorn

move new object into a randomly chosen place in the world

set starting energy according to objectrsquos type

onKill

if ordfCreaturesordm object died save its performance in ordfGenotypesordm (possiblycreating a new genotype) If there are too many genotypes in ordfGenotypesordmremove one

onFoodCollision

send energy portion from ordfFoodordm object to ordfCreatureordm object

4 Additional toolsMany research works concern studies of evolutionary processes theirdynamics and efregciency Various measures and methods have been developedin order to be able to analyze evolution complexity and interaction in theobserved systems Other works try to understand behaviors of artiregcialcreatures regarding them as subjects of survey rather than black boxes withassigned regtness and performance

Artiregcial life systems especially those applied to evolutionary robotics anddesign (Bentley 1999 Funes and Pollack 1998 Lipson and Pollack 2000) arequite complex and it is difregcult to understand the behavior of existing agents in

Figure 6A fragment of softwarewindow with experimentparameters

K3212

164

detail The only way is to observe them carefully and use human intelligence todraw conclusions Usually the behavior of such agents is non-deterministicand their control systems are sophisticated often coupled with morphologyand very strongly connected functionally (Lund et al 1997)

Thus for the purpose of studying behaviors and populations of individualsone needs high-level intelligent support tools (KomosinAcircski and Rotaru-Varga2000) It is not likely that automatic tools will soon be able to produceunderstandable non-trivial explanations of sophisticated artiregcial agentsNonetheless their role and help cannot be ignored Even simple automaticsupport is of great relevance to a human which becomes obvious afterspending hours on investigating relatively simple artiregcial creatures It maybe that in the future some advanced analysis methods developed withinartiregcial life methodology will be useful for real life studies biology andmedicine

One of the main purposes of the Framsticks system is to allow creating andtesting such tools and procedures and to develop methodology needed fortheir use Realistic artiregcial life environments are the right place for suchresearch

41 Clustering of similar individualsSimilarity seems to be a simple property However automatic measures ofsimilarity can help in observation of regularities groups of related individualsetc Similarity can be identireged in many ways including aspects of morphology(body) brain size function behavior performance regtness etc Whateverderegnition is used automatic measure of similarity can be useful for

optimization to introduce artiregcial niches by modiregcation of regtnessvalues (Goldberg 1989)

studies of evolutionary processes and the structure of populations ofindividuals

studies of functionbehavior of agents

reduction of the number of agents to a small subset of interesting diverseunique individuals

inferring dendrograms (and hopefully phylogenetic trees) based ondistances between organisms

For Framsticks we constructed a heuristic method that is able to estimate thedegree of similarity of the two individuals This method treats body as a graph(with material points as vertices and joints as edges) and then tries to matchtwo body structures based on the degrees of vertices as the main piece ofinformation For a more detailed description of this method see KomosinAcircskiet al (2001) and an example is presented in Section 5

The Framstickssystem

165

42 History of evolutionIn real life although we are able to trace genetic relationships within existingcreatures we do not know what happens during mutation and crossing over oftheir genomes Moreover we cannot trace genetic relations in a longer scale andhigher number of individuals

In Framsticks it is possible not only to remember all parent-childrelationships but also to estimate genetic shares of related individuals (howmany genes have mutated or have been exchanged) This allows to draw a realtree of evolution as shown in Figure 7 Vertical axis is time horizontal oneremacrects local degree of genetic dissimilarity (between a pair of individuals)Vertices in the tree are single individuals

43 Fuzzy controlFuzzy control nowadays has become a very popular method of controlling inmany domains of our life washing machines video cameras ABS air-condition etc It is also used for controlling non-linear fast-changing processeswhere quick decisions are more important than exact ones (Yager and Filev1994) Fuzzy control

Figure 7The real tree ofevolution Top singleancestor and beginningof time Black linesrepresent mutationswhite ones arecrossovers

K3212

166

allows for linguistic variables ordfDrive fastordm is much more macrexible thanordfdrive with speed 80-100 kmhordm In fuzzy approach we would say 79 kmhis still fast with degree 95 percent

better comprehension for humans plusmn fuzzy rule if X is Big and Y is Smallthen Z is Medium is easier to understand than crispy one if X is between3222 and 4332 and Y is less than 52 then Z is 192

more macruent ordfnaturalordm change of system states plusmn for example gradualslowing down of a car

For such reasons fuzzy control is developed in Framsticks Therefore evolvedFramsticksrsquo brains may be more understandable for a human Fuzzy controlsimilarly to neural networks can also cope with uncertainty of information plusmnwhen the process is compound depends on lottery events or the measurementis burden with error Fuzzy approach can manage this inconvenience bygeneralization of information To incorporate this approach in Framsticksneural network model three additional types of neurons are needed

difference neuron plusmn to compare current value of the sensor to the previousone

rule neuron plusmn to represent a fuzzy rule

aggregate neuron plusmn to gather answers of the rule neurons and calculatecrisp output

The two sample fuzzy rules are

IF gyroscope is big+ AND Dgyroscope is small+

THEN bend muscle medium 2

IF gyroscope is medium 2 AND Dgyroscope is zero+

THEN bend muscle big+

where gyroscope is the measurement taken from the gyroscope receptorDgyroscope is the change in its value and linguistic terms ordfbig+ordm etccorrespond to fuzzy intervals of values The representation of these two rules inthe Framsticks neural network is shown in Figure 8

44 Helper softwareTogether with continuous development of extensions to the Framsticks systemitself there are some external tools created Fred is a visual Framsticks Editordeveloped in JAVA which allows to design a creature (or a structure) in a user-friendly way Framsticks Experimentation Center is an Internet database ofgenotypes and experiment deregnitionsproposals which can be browseddownloaded and uploaded

The Framstickssystem

167

5 Research experiments51 Comparison of genotype encodingsThere are a number of studies of the evolution of simulated creatures equippedwith realistic physical behavior In such systems the use of a physicalsimulation layer implements a complex genotype-regtness relationshipPhysical interactions between body parts the coupling between control andphysical body and interactions during body development can all add a level ofindirection between the genotype and regtness The complexity of the genotype-regtness relationship offers a potential for rich evolutionary dynamics

The most important element of the genotype-to-regtness relationship is thegenotype-to-phenotype mapping or genotype encoding There is no obvioussimple way to encode a complex phenotype plusmn which consists of a variable-sizestructured body and a matching control system plusmn into a simpler genotypeMoreover an evolutionary algorithm can perform poorly when using a certaingenotype encoding and better when using others for reasons not yetimmediately obvious The employed genotype encoding can have a signiregcanteffect on the performance of the evolution

The Framsticks system has been used as the context of analysis of variousgenotype encodings The performance of the three encodings described inSection 3 was compared in their optimization tasks passive and active heightand velocity maximization (KomosinAcircski and Rotaru-Varga 2001) Thesolutions produced by evolution are considered to be successful for the giventasks in all three cases However there were some important differences in thedegree of success The f 0 encoding performed worse than the two higher-level

Figure 8Two fuzzy rules encodedas a Framsticks neuralnetwork

K3212

168

encodings The most important differences between these encodings are thatf 0 has a minimal bias and is unrestrictive while the higher-level encodings (f1and f4) restrict the search space and introduce a strong bias towardsstructured phenotypes Based on these results we conclude that a morestructured genotype encoding with genetic operators working on a higherlevel is beneregcial in the evolution of 3D agents The presence of a bias towardsstructured phenotypes can overcome the apparent limitation that entire regionsof the search space are not accessible by the search This bias may be useful insome applications (engineering and robotics for example) The signiregcantinmacruence of a chosen encoding can be clearly seen in the obtained agents thosewith f 0 encoding displayed neither order nor structure The two encodingsrestricting morphology to a tree produced more clear constructions and fordevelopmental encoding segmentation and modularity could be observed(Figure 9)

Figure 9Representative agents

for various geneticencodings and height

maximization task

The Framstickssystem

169

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

detail The only way is to observe them carefully and use human intelligence todraw conclusions Usually the behavior of such agents is non-deterministicand their control systems are sophisticated often coupled with morphologyand very strongly connected functionally (Lund et al 1997)

Thus for the purpose of studying behaviors and populations of individualsone needs high-level intelligent support tools (KomosinAcircski and Rotaru-Varga2000) It is not likely that automatic tools will soon be able to produceunderstandable non-trivial explanations of sophisticated artiregcial agentsNonetheless their role and help cannot be ignored Even simple automaticsupport is of great relevance to a human which becomes obvious afterspending hours on investigating relatively simple artiregcial creatures It maybe that in the future some advanced analysis methods developed withinartiregcial life methodology will be useful for real life studies biology andmedicine

One of the main purposes of the Framsticks system is to allow creating andtesting such tools and procedures and to develop methodology needed fortheir use Realistic artiregcial life environments are the right place for suchresearch

41 Clustering of similar individualsSimilarity seems to be a simple property However automatic measures ofsimilarity can help in observation of regularities groups of related individualsetc Similarity can be identireged in many ways including aspects of morphology(body) brain size function behavior performance regtness etc Whateverderegnition is used automatic measure of similarity can be useful for

optimization to introduce artiregcial niches by modiregcation of regtnessvalues (Goldberg 1989)

studies of evolutionary processes and the structure of populations ofindividuals

studies of functionbehavior of agents

reduction of the number of agents to a small subset of interesting diverseunique individuals

inferring dendrograms (and hopefully phylogenetic trees) based ondistances between organisms

For Framsticks we constructed a heuristic method that is able to estimate thedegree of similarity of the two individuals This method treats body as a graph(with material points as vertices and joints as edges) and then tries to matchtwo body structures based on the degrees of vertices as the main piece ofinformation For a more detailed description of this method see KomosinAcircskiet al (2001) and an example is presented in Section 5

The Framstickssystem

165

42 History of evolutionIn real life although we are able to trace genetic relationships within existingcreatures we do not know what happens during mutation and crossing over oftheir genomes Moreover we cannot trace genetic relations in a longer scale andhigher number of individuals

In Framsticks it is possible not only to remember all parent-childrelationships but also to estimate genetic shares of related individuals (howmany genes have mutated or have been exchanged) This allows to draw a realtree of evolution as shown in Figure 7 Vertical axis is time horizontal oneremacrects local degree of genetic dissimilarity (between a pair of individuals)Vertices in the tree are single individuals

43 Fuzzy controlFuzzy control nowadays has become a very popular method of controlling inmany domains of our life washing machines video cameras ABS air-condition etc It is also used for controlling non-linear fast-changing processeswhere quick decisions are more important than exact ones (Yager and Filev1994) Fuzzy control

Figure 7The real tree ofevolution Top singleancestor and beginningof time Black linesrepresent mutationswhite ones arecrossovers

K3212

166

allows for linguistic variables ordfDrive fastordm is much more macrexible thanordfdrive with speed 80-100 kmhordm In fuzzy approach we would say 79 kmhis still fast with degree 95 percent

better comprehension for humans plusmn fuzzy rule if X is Big and Y is Smallthen Z is Medium is easier to understand than crispy one if X is between3222 and 4332 and Y is less than 52 then Z is 192

more macruent ordfnaturalordm change of system states plusmn for example gradualslowing down of a car

For such reasons fuzzy control is developed in Framsticks Therefore evolvedFramsticksrsquo brains may be more understandable for a human Fuzzy controlsimilarly to neural networks can also cope with uncertainty of information plusmnwhen the process is compound depends on lottery events or the measurementis burden with error Fuzzy approach can manage this inconvenience bygeneralization of information To incorporate this approach in Framsticksneural network model three additional types of neurons are needed

difference neuron plusmn to compare current value of the sensor to the previousone

rule neuron plusmn to represent a fuzzy rule

aggregate neuron plusmn to gather answers of the rule neurons and calculatecrisp output

The two sample fuzzy rules are

IF gyroscope is big+ AND Dgyroscope is small+

THEN bend muscle medium 2

IF gyroscope is medium 2 AND Dgyroscope is zero+

THEN bend muscle big+

where gyroscope is the measurement taken from the gyroscope receptorDgyroscope is the change in its value and linguistic terms ordfbig+ordm etccorrespond to fuzzy intervals of values The representation of these two rules inthe Framsticks neural network is shown in Figure 8

44 Helper softwareTogether with continuous development of extensions to the Framsticks systemitself there are some external tools created Fred is a visual Framsticks Editordeveloped in JAVA which allows to design a creature (or a structure) in a user-friendly way Framsticks Experimentation Center is an Internet database ofgenotypes and experiment deregnitionsproposals which can be browseddownloaded and uploaded

The Framstickssystem

167

5 Research experiments51 Comparison of genotype encodingsThere are a number of studies of the evolution of simulated creatures equippedwith realistic physical behavior In such systems the use of a physicalsimulation layer implements a complex genotype-regtness relationshipPhysical interactions between body parts the coupling between control andphysical body and interactions during body development can all add a level ofindirection between the genotype and regtness The complexity of the genotype-regtness relationship offers a potential for rich evolutionary dynamics

The most important element of the genotype-to-regtness relationship is thegenotype-to-phenotype mapping or genotype encoding There is no obvioussimple way to encode a complex phenotype plusmn which consists of a variable-sizestructured body and a matching control system plusmn into a simpler genotypeMoreover an evolutionary algorithm can perform poorly when using a certaingenotype encoding and better when using others for reasons not yetimmediately obvious The employed genotype encoding can have a signiregcanteffect on the performance of the evolution

The Framsticks system has been used as the context of analysis of variousgenotype encodings The performance of the three encodings described inSection 3 was compared in their optimization tasks passive and active heightand velocity maximization (KomosinAcircski and Rotaru-Varga 2001) Thesolutions produced by evolution are considered to be successful for the giventasks in all three cases However there were some important differences in thedegree of success The f 0 encoding performed worse than the two higher-level

Figure 8Two fuzzy rules encodedas a Framsticks neuralnetwork

K3212

168

encodings The most important differences between these encodings are thatf 0 has a minimal bias and is unrestrictive while the higher-level encodings (f1and f4) restrict the search space and introduce a strong bias towardsstructured phenotypes Based on these results we conclude that a morestructured genotype encoding with genetic operators working on a higherlevel is beneregcial in the evolution of 3D agents The presence of a bias towardsstructured phenotypes can overcome the apparent limitation that entire regionsof the search space are not accessible by the search This bias may be useful insome applications (engineering and robotics for example) The signiregcantinmacruence of a chosen encoding can be clearly seen in the obtained agents thosewith f 0 encoding displayed neither order nor structure The two encodingsrestricting morphology to a tree produced more clear constructions and fordevelopmental encoding segmentation and modularity could be observed(Figure 9)

Figure 9Representative agents

for various geneticencodings and height

maximization task

The Framstickssystem

169

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

42 History of evolutionIn real life although we are able to trace genetic relationships within existingcreatures we do not know what happens during mutation and crossing over oftheir genomes Moreover we cannot trace genetic relations in a longer scale andhigher number of individuals

In Framsticks it is possible not only to remember all parent-childrelationships but also to estimate genetic shares of related individuals (howmany genes have mutated or have been exchanged) This allows to draw a realtree of evolution as shown in Figure 7 Vertical axis is time horizontal oneremacrects local degree of genetic dissimilarity (between a pair of individuals)Vertices in the tree are single individuals

43 Fuzzy controlFuzzy control nowadays has become a very popular method of controlling inmany domains of our life washing machines video cameras ABS air-condition etc It is also used for controlling non-linear fast-changing processeswhere quick decisions are more important than exact ones (Yager and Filev1994) Fuzzy control

Figure 7The real tree ofevolution Top singleancestor and beginningof time Black linesrepresent mutationswhite ones arecrossovers

K3212

166

allows for linguistic variables ordfDrive fastordm is much more macrexible thanordfdrive with speed 80-100 kmhordm In fuzzy approach we would say 79 kmhis still fast with degree 95 percent

better comprehension for humans plusmn fuzzy rule if X is Big and Y is Smallthen Z is Medium is easier to understand than crispy one if X is between3222 and 4332 and Y is less than 52 then Z is 192

more macruent ordfnaturalordm change of system states plusmn for example gradualslowing down of a car

For such reasons fuzzy control is developed in Framsticks Therefore evolvedFramsticksrsquo brains may be more understandable for a human Fuzzy controlsimilarly to neural networks can also cope with uncertainty of information plusmnwhen the process is compound depends on lottery events or the measurementis burden with error Fuzzy approach can manage this inconvenience bygeneralization of information To incorporate this approach in Framsticksneural network model three additional types of neurons are needed

difference neuron plusmn to compare current value of the sensor to the previousone

rule neuron plusmn to represent a fuzzy rule

aggregate neuron plusmn to gather answers of the rule neurons and calculatecrisp output

The two sample fuzzy rules are

IF gyroscope is big+ AND Dgyroscope is small+

THEN bend muscle medium 2

IF gyroscope is medium 2 AND Dgyroscope is zero+

THEN bend muscle big+

where gyroscope is the measurement taken from the gyroscope receptorDgyroscope is the change in its value and linguistic terms ordfbig+ordm etccorrespond to fuzzy intervals of values The representation of these two rules inthe Framsticks neural network is shown in Figure 8

44 Helper softwareTogether with continuous development of extensions to the Framsticks systemitself there are some external tools created Fred is a visual Framsticks Editordeveloped in JAVA which allows to design a creature (or a structure) in a user-friendly way Framsticks Experimentation Center is an Internet database ofgenotypes and experiment deregnitionsproposals which can be browseddownloaded and uploaded

The Framstickssystem

167

5 Research experiments51 Comparison of genotype encodingsThere are a number of studies of the evolution of simulated creatures equippedwith realistic physical behavior In such systems the use of a physicalsimulation layer implements a complex genotype-regtness relationshipPhysical interactions between body parts the coupling between control andphysical body and interactions during body development can all add a level ofindirection between the genotype and regtness The complexity of the genotype-regtness relationship offers a potential for rich evolutionary dynamics

The most important element of the genotype-to-regtness relationship is thegenotype-to-phenotype mapping or genotype encoding There is no obvioussimple way to encode a complex phenotype plusmn which consists of a variable-sizestructured body and a matching control system plusmn into a simpler genotypeMoreover an evolutionary algorithm can perform poorly when using a certaingenotype encoding and better when using others for reasons not yetimmediately obvious The employed genotype encoding can have a signiregcanteffect on the performance of the evolution

The Framsticks system has been used as the context of analysis of variousgenotype encodings The performance of the three encodings described inSection 3 was compared in their optimization tasks passive and active heightand velocity maximization (KomosinAcircski and Rotaru-Varga 2001) Thesolutions produced by evolution are considered to be successful for the giventasks in all three cases However there were some important differences in thedegree of success The f 0 encoding performed worse than the two higher-level

Figure 8Two fuzzy rules encodedas a Framsticks neuralnetwork

K3212

168

encodings The most important differences between these encodings are thatf 0 has a minimal bias and is unrestrictive while the higher-level encodings (f1and f4) restrict the search space and introduce a strong bias towardsstructured phenotypes Based on these results we conclude that a morestructured genotype encoding with genetic operators working on a higherlevel is beneregcial in the evolution of 3D agents The presence of a bias towardsstructured phenotypes can overcome the apparent limitation that entire regionsof the search space are not accessible by the search This bias may be useful insome applications (engineering and robotics for example) The signiregcantinmacruence of a chosen encoding can be clearly seen in the obtained agents thosewith f 0 encoding displayed neither order nor structure The two encodingsrestricting morphology to a tree produced more clear constructions and fordevelopmental encoding segmentation and modularity could be observed(Figure 9)

Figure 9Representative agents

for various geneticencodings and height

maximization task

The Framstickssystem

169

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

allows for linguistic variables ordfDrive fastordm is much more macrexible thanordfdrive with speed 80-100 kmhordm In fuzzy approach we would say 79 kmhis still fast with degree 95 percent

better comprehension for humans plusmn fuzzy rule if X is Big and Y is Smallthen Z is Medium is easier to understand than crispy one if X is between3222 and 4332 and Y is less than 52 then Z is 192

more macruent ordfnaturalordm change of system states plusmn for example gradualslowing down of a car

For such reasons fuzzy control is developed in Framsticks Therefore evolvedFramsticksrsquo brains may be more understandable for a human Fuzzy controlsimilarly to neural networks can also cope with uncertainty of information plusmnwhen the process is compound depends on lottery events or the measurementis burden with error Fuzzy approach can manage this inconvenience bygeneralization of information To incorporate this approach in Framsticksneural network model three additional types of neurons are needed

difference neuron plusmn to compare current value of the sensor to the previousone

rule neuron plusmn to represent a fuzzy rule

aggregate neuron plusmn to gather answers of the rule neurons and calculatecrisp output

The two sample fuzzy rules are

IF gyroscope is big+ AND Dgyroscope is small+

THEN bend muscle medium 2

IF gyroscope is medium 2 AND Dgyroscope is zero+

THEN bend muscle big+

where gyroscope is the measurement taken from the gyroscope receptorDgyroscope is the change in its value and linguistic terms ordfbig+ordm etccorrespond to fuzzy intervals of values The representation of these two rules inthe Framsticks neural network is shown in Figure 8

44 Helper softwareTogether with continuous development of extensions to the Framsticks systemitself there are some external tools created Fred is a visual Framsticks Editordeveloped in JAVA which allows to design a creature (or a structure) in a user-friendly way Framsticks Experimentation Center is an Internet database ofgenotypes and experiment deregnitionsproposals which can be browseddownloaded and uploaded

The Framstickssystem

167

5 Research experiments51 Comparison of genotype encodingsThere are a number of studies of the evolution of simulated creatures equippedwith realistic physical behavior In such systems the use of a physicalsimulation layer implements a complex genotype-regtness relationshipPhysical interactions between body parts the coupling between control andphysical body and interactions during body development can all add a level ofindirection between the genotype and regtness The complexity of the genotype-regtness relationship offers a potential for rich evolutionary dynamics

The most important element of the genotype-to-regtness relationship is thegenotype-to-phenotype mapping or genotype encoding There is no obvioussimple way to encode a complex phenotype plusmn which consists of a variable-sizestructured body and a matching control system plusmn into a simpler genotypeMoreover an evolutionary algorithm can perform poorly when using a certaingenotype encoding and better when using others for reasons not yetimmediately obvious The employed genotype encoding can have a signiregcanteffect on the performance of the evolution

The Framsticks system has been used as the context of analysis of variousgenotype encodings The performance of the three encodings described inSection 3 was compared in their optimization tasks passive and active heightand velocity maximization (KomosinAcircski and Rotaru-Varga 2001) Thesolutions produced by evolution are considered to be successful for the giventasks in all three cases However there were some important differences in thedegree of success The f 0 encoding performed worse than the two higher-level

Figure 8Two fuzzy rules encodedas a Framsticks neuralnetwork

K3212

168

encodings The most important differences between these encodings are thatf 0 has a minimal bias and is unrestrictive while the higher-level encodings (f1and f4) restrict the search space and introduce a strong bias towardsstructured phenotypes Based on these results we conclude that a morestructured genotype encoding with genetic operators working on a higherlevel is beneregcial in the evolution of 3D agents The presence of a bias towardsstructured phenotypes can overcome the apparent limitation that entire regionsof the search space are not accessible by the search This bias may be useful insome applications (engineering and robotics for example) The signiregcantinmacruence of a chosen encoding can be clearly seen in the obtained agents thosewith f 0 encoding displayed neither order nor structure The two encodingsrestricting morphology to a tree produced more clear constructions and fordevelopmental encoding segmentation and modularity could be observed(Figure 9)

Figure 9Representative agents

for various geneticencodings and height

maximization task

The Framstickssystem

169

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

5 Research experiments51 Comparison of genotype encodingsThere are a number of studies of the evolution of simulated creatures equippedwith realistic physical behavior In such systems the use of a physicalsimulation layer implements a complex genotype-regtness relationshipPhysical interactions between body parts the coupling between control andphysical body and interactions during body development can all add a level ofindirection between the genotype and regtness The complexity of the genotype-regtness relationship offers a potential for rich evolutionary dynamics

The most important element of the genotype-to-regtness relationship is thegenotype-to-phenotype mapping or genotype encoding There is no obvioussimple way to encode a complex phenotype plusmn which consists of a variable-sizestructured body and a matching control system plusmn into a simpler genotypeMoreover an evolutionary algorithm can perform poorly when using a certaingenotype encoding and better when using others for reasons not yetimmediately obvious The employed genotype encoding can have a signiregcanteffect on the performance of the evolution

The Framsticks system has been used as the context of analysis of variousgenotype encodings The performance of the three encodings described inSection 3 was compared in their optimization tasks passive and active heightand velocity maximization (KomosinAcircski and Rotaru-Varga 2001) Thesolutions produced by evolution are considered to be successful for the giventasks in all three cases However there were some important differences in thedegree of success The f 0 encoding performed worse than the two higher-level

Figure 8Two fuzzy rules encodedas a Framsticks neuralnetwork

K3212

168

encodings The most important differences between these encodings are thatf 0 has a minimal bias and is unrestrictive while the higher-level encodings (f1and f4) restrict the search space and introduce a strong bias towardsstructured phenotypes Based on these results we conclude that a morestructured genotype encoding with genetic operators working on a higherlevel is beneregcial in the evolution of 3D agents The presence of a bias towardsstructured phenotypes can overcome the apparent limitation that entire regionsof the search space are not accessible by the search This bias may be useful insome applications (engineering and robotics for example) The signiregcantinmacruence of a chosen encoding can be clearly seen in the obtained agents thosewith f 0 encoding displayed neither order nor structure The two encodingsrestricting morphology to a tree produced more clear constructions and fordevelopmental encoding segmentation and modularity could be observed(Figure 9)

Figure 9Representative agents

for various geneticencodings and height

maximization task

The Framstickssystem

169

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

encodings The most important differences between these encodings are thatf 0 has a minimal bias and is unrestrictive while the higher-level encodings (f1and f4) restrict the search space and introduce a strong bias towardsstructured phenotypes Based on these results we conclude that a morestructured genotype encoding with genetic operators working on a higherlevel is beneregcial in the evolution of 3D agents The presence of a bias towardsstructured phenotypes can overcome the apparent limitation that entire regionsof the search space are not accessible by the search This bias may be useful insome applications (engineering and robotics for example) The signiregcantinmacruence of a chosen encoding can be clearly seen in the obtained agents thosewith f 0 encoding displayed neither order nor structure The two encodingsrestricting morphology to a tree produced more clear constructions and fordevelopmental encoding segmentation and modularity could be observed(Figure 9)

Figure 9Representative agents

for various geneticencodings and height

maximization task

The Framstickssystem

169

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

52 Automatic optimization versus human designDesigning agents by hand is a very complex process in professionalapplications it requires planning and extensive knowledge about how thecontrol system receptors and effectors work as well as knowledge about thesimulator Designing neural networks for control by hand is especially difregcultand tedious For this reason human-built agents usually have lower regtnessthan agents produced by evolution However human creations are ofteninteresting qualitatively Human designs have such properties as explicitpurpose elegance simplicity (minimum of means) and often symmetry andmodularity These features are opposed to evolutionary results which arecharacterized by hidden purpose complexity implicit and very stronginterdependencies between parts as well as redundancy and randomness(KomosinAcircski and Rotaru-Varga 2001)

The difregcult process of designing neural networks can be circumvented by ahybrid solution bodies can be hand-constructed and control structures evolvedfor it This approach can yield interesting creatures (Adamatzky 2000KomosinAcircski 2000 KomosinAcircski and Rotaru-Varga 2000 KomosinAcircski andUlatowski 1997) often resembling in behavior creatures found in nature(Ijspeert 2000)

53 Clustering with similarity measureThe similarity measure outlined in Section 4 allowed us to perform variousexperiments (KomosinAcircski et al 2001) Figure 10 shows the results of applicationof UPGMA clustering method to ten individuals from the height maximizationtask The reggure shows their morphologies together with the clustering tree

It can be seen that the three big organisms are in a single distinct clusterThey are similar in size but different in structure so the distance in-betweenthem is high Moreover the measure captured also functional similarity (hp_13 6 9 7) plusmn all these agents have a single stick upwards and a similar base Theagents hp_0 and hp_4 are of medium size but certainly closer to the smallorganisms group than to the big ones They are also similar in structure that iswhy they constitute a separate cluster

The similarity measure is very helpful for study and analysis of groups ofindividuals Manual work of classiregcation of the agents shown on Figure 9yielded similar results but it was a very mundane and time-consumingprocess It also lacked objectivism and accuracy which are properties of theautomatic procedure

54 Other possibilitiesConsidering open architecture of the Framsticks system many possibilitiesexist of deregning diverse genetic representations experimental setupsinteraction rules and environments (see Section 3) Most obvious ideas

K3212

170

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

include coevolution of individuals in a population predatorprey relationshipsmultiple gene pools and populations and their specialization

Some of the possible experiments related to biology include introducinggeographical constraints and then looking at the differences in clustersobtained after a given period of time or studying two or more populations ofmuch differing sizes The latter under geographical constraints could be usedto simulate and understand speciation

Refer Mandik (2002a b) for a number of interesting experiments regardingevolutionary and neuro-computational bases of the emergence of complexforms of cognition and a discussion about semantics of evolved neuralnetworks perception and memory

Figure 10The clustering tree for

ten best individuals fromthe height maximization

task

The Framstickssystem

171

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

6 SummaryThis paper was devoted to Framsticks a general tool for modeling simulationand optimization of virtual creatures Although the system is versatile andcomplex one should remember that it may be simplireged when some featuresare not needed For example control systems can be neglected if only staticstructures are to be considered Genetic encoding may only allow for two-dimensional structures if 3D is not required Local optimization frameworkmay be used if the task does not require evolutionary algorithms etc

Complexity is useless when it cannot be understood or used This is whyFramsticks software tries to present information in a human-friendly and clearway It is not only employed in research experiments but is also popular ineducation The system is used by computer scientists biologists roboticistsand other scientists but also by students and laypeople of various age

Note

1 However there are plans to integrate a very accurate simulation engine into the systemwhen it is required such engines are used in some works (Taylor and Massey 2001)

References

Adamatzky A (2000) ordfSoftware review Framsticksordm Kybernetes The International Journal ofSystems and Cybernetics Vol 29 No 9-10 pp 1344-51

Bentley P (1999) Evolutionary Design by Computers Morgan Kaufmann

Funes P and Pollack JB (1998) ordfEvolutionary body building adaptive physical designs forrobotsordm Artiregcial Life Vol 4 No 4 Autumn pp 337-57

Goldberg DE (1989) Genetic Algorithms in Search Optimization and Machine LearningAddison-Wesley Reading MA USA

Gruau F Whitley D and Pyeatt L (1996) ordfA comparison between cellular encoding and directencoding for genetic neural networksordm in Koza JR Goldberg DE Fogel DB and RioloRR (Eds) Proceedings of the First Annual Conference Genetic Programming 1996 MITPress Cambridge MA pp 81-9

Ijspeert AJ (2000) ordfA 3-D biomechanical model of the salamanderordm in Heudin J-C (Ed)Proceedings of 2nd International Conference on Virtual Worlds (VW2000) LNAI 1834July 2000 Springer-Verlag Paris France pp 225-34

KomosinAcircski M (2000) ordfThe world of Framsticks simulation evolution interactionordm in HeudinJ-C (Ed) Virtual Worlds Lecture Notes in Artiregcial Intelligence 1834 Springer-Verlagpp 214-24

KomosinAcircski M and Rotaru-Varga A (2000) ordfFrom directed to open-ended evolution ina complex simulation modelordm in Bedau MA McCaskill JS Packard NH andRasmussen S (Eds) Artiregcial Life VII MIT Press pp 293-9

KomosinAcircski M and Rotaru-Varga A (2001) ordfComparison of different genotype encodings forsimulated 3D agentsordm Artiregcial Life Journal Vol 7 No 4 Fall pp 395-418

KomosinAcircski M and Ulatowski S (1997) Framsticks Web Site httpwwwframsalifepl

KomosinAcirc ski M Koczyk G and Kubiak M (2001) ordfOn estimating similarity of artiregcial andreal organismsordm Theory in Biosciences Vol 120 pp 271-86

K3212

172

Lipson H and Pollack JB (2000) ordfAutomatic design and manufacture of robotic lifeformsordmNature Vol 406 No 6799 pp 974-8

Lund HH Hallam J and Lee W-P (1997) ordfEvolving robot morphologyordm Proceedings of IEEE4th International Conference on Evolutionary Computation (Invited paper) IEEE Press NJ

Mandik P (2002a) ordfSynthetic neuroethologyordm Metaphilosophy Vol 33 No 12 pp 11-29 httpwwwwpunjeducohssphilosophyfacultymandikpaperssynthneurpdf

Mandik P (2002b) ordfVarieties of representation in evolved and embodied neural networksordmWilliam Paterson University Cognitive Science Technical Report 2002-03 httpwwwwpunjeducohssphilosophyfacultymandikpapersvreennpdf

Taylor T and Massey C (2001) ordfRecent developments in the evolution of morphologies andcontrollers for physically simulated creaturesordm Artiregcial Life Vol 7 No 1 Winterpp 77-88

Ulatowski S (2000) Framsticks GDK (Genotype Development Kit) httpwwwframsalifepldev

Yager RR and Filev DP (1994) Foundations of Fuzzy Control Wiley New York

The Framstickssystem

173