30
@GeneticIO DevoxxUK 2015 #DevoxxUK Devoxx UK – 17th – 19th June, 2015 Genetic. io Yves Corsaire Julien Sebrien [email protected] Building Scalable Applications That Implement Genetic Algorithms for free

Building Scalable Applications That Implement Genetic Algorithms with Genetic.io

Embed Size (px)

Citation preview

@GeneticIO DevoxxUK 2015 #DevoxxUK

Devoxx UK – 17th – 19th June, 2015

Genetic.io

Yves CorsaireJulien Sebrien

[email protected]

Building Scalable Applications That Implement Genetic Algorithms

for free

@GeneticIO DevoxxUK 2015 #DevoxxUK

Genetic Algorithm?

A genetic algorithm (GA) is a search technique used to find exact or approximate solutions to optimization and search problems

Wikipedia ;)

@GeneticIO DevoxxUK 2015 #DevoxxUK

Sure…

@GeneticIO DevoxxUK 2015 #DevoxxUK

Show me an real life

example!

@GeneticIO DevoxxUK 2015 #DevoxxUK

Satellite orbit selection!

No kidding ! http://goo.gl/eauC32

philae

tchoury 

@GeneticIO DevoxxUK 2015 #DevoxxUK

What do I need?

• A genetic encoding of each solutionEach individual holds information already present in the problem (e.g. velocity, weights), and information that needs to be computed or derived.

• A selection method : Determines how individuals are retained between each generation (Fitness Proportionate, Tournament, Rank).

• A fitness functionEvaluates the "goodness" of each individual (candidate solution).

.

@GeneticIO DevoxxUK 2015 #DevoxxUK

Evolution process

Build Initial Population Evaluation

Terminated?

Mutation

Crossover

Selection

Done!

Yes

No

@GeneticIO DevoxxUK 2015 #DevoxxUK

Problem Modeling

find a route to « tchoury » comet

a sequence of n chromosomes, each chromosome being a unitary 2D movement

the closest from the comet a candidate is, the highest is its rate.

f(x) = 1/ R (R=remaining distance from comet)

• Requirements

• Candidate Implementation

• Fitness Function

@GeneticIO DevoxxUK 2015 #DevoxxUK

Selection

Multiple ways to select individuals exist : Fitness Proportionate, Tournament, Rank, etc.

Tournament Example

1.Randomly select two individuals from the population.

2.Generate a random value to decide whether to select the fitter individual or the weaker one.

3.Add the chosen individual to the current selection. The two individuals are then returned to the original population and can be selected again.

This type of selection tends to favor the fitter one more as the tournament size increases.

@GeneticIO DevoxxUK 2015 #DevoxxUK

Crossover

NE N E NE S W SW E

E NE N NW W E E NW

one crossover point one crossover point

parent 1 parent 2

child 1 child 2

NE N E NE S E E NW

E NE N NW W W SW E

@GeneticIO DevoxxUK 2015 #DevoxxUK

MutationObjective : inject diversity into the population, prompting the genetic algorithms to explore new solutions and as such lower the risk of being trapped in a local optimum

Description : generate a random number. If it’s less than the chosen probability of mutation (ie. Mutation Rate), then mutation takes place, else aborts.

Graph from : http://www.codeproject.com/Articles/707505/Genetic-Algorithms-Demystified

child 2

… W …

… E …

@GeneticIO DevoxxUK 2015 #DevoxxUK

Evaluation

Fitness Function = remaining distance

tchoury 

philaephilae

philae

philaephilae

philae

philae

philae

philae

philae

philaephilae

@GeneticIO DevoxxUK 2015 #DevoxxUK

Terminated?

The algorithm terminates once at least one stopping criteria is met:

1. A pre-determined number of iterations or generations is reached.

2. A candidate that meets or exceeds a target fitness value is found.

3. The algorithm runs for a too much amount of time.

@GeneticIO DevoxxUK 2015 #DevoxxUK

What choice do we have?

@GeneticIO DevoxxUK 2015 #DevoxxUK

Current librairies

• Numerous exist (JGAP, ECJ, EpochX) in multiple languages (Java, C++, Python, etc.)

• None of them actually :

– Provide Modern Web UIs…

– Work with multiple languages…

– Run on distributed architectures…

@GeneticIO DevoxxUK 2015 #DevoxxUK

Until now;)

@GeneticIO DevoxxUK 2015 #DevoxxUK

Easy Set Up• Install Genetic.io using provided scripts (windows, unix, mac)

• Create user accounts

• Import your data using various importation methods (local file, upload, remote database, etc.)

• Configure your job parameters (evolution engine type, classes dependencies, termination conditions, etc.)

• Set up fitness computation parallelism method (spark, jppf, multitheading)

• Start and see realtime results!

@GeneticIO DevoxxUK 2015 #DevoxxUK

Software Stack

Web Application Server

Web Client

HTTPWebSocket

Database Cluster

Grid Computing Cluster

Batch

CQL

CQL

Wire

Wire Data Computing Cluster

@GeneticIO DevoxxUK 2015 #DevoxxUK

Infrastructure

start genetic jobs, get re

al-time progress

Store/Loadaccounts, datasources, jobs conf.

compute candidates fitnesses

Load

can

dida

tes

(with

dat

a lo

cality

)

Load jobs configuration, store progress

@GeneticIO DevoxxUK 2015 #DevoxxUK

Data Importation

@GeneticIO DevoxxUK 2015 #DevoxxUK

Data conversion

@GeneticIO DevoxxUK 2015 #DevoxxUK

Fitness Calculator

public class PhilaeFitnessCalculator implements FitnessCalculator{@Overridepublic double calculate(IndividualBean bean, ...) {

double remainingDistance = computeRemainingDistance(bean);

return 1/remainingDistance;}@Overridepublic boolean isNatural() {

return true;}

private double computeRemainingDistance(IndividualBean bean){return ...;

}}

@GeneticIO DevoxxUK 2015 #DevoxxUK

Crossover Operatorpublic class PhilaeCrossoverOperator implements CrossoverOperator {

...@Overridepublic List<IndividualBean> crossover(IndividualBean firstParent, IndividualBean

secondParent, Random rng) {IndividualBean firstChild = new IndividualBean(firstParent);IndividualBean secondChild = new IndividualBean(secondParent);int numberOfCrossoverPoints = getNbCrossoverPoints(); for (int i = 0; i < numberOfCrossoverPoints; i++){ int max = Math.min(firstChild.getNbAttributes(),

secondChild.getNbAttributes()); if (max > 1){

int crossoverIndex = (1 + rng.nextInt(max - 1)); for (int j = 0; j < crossoverIndex; j++){ Object temp = firstChild.getValue(j); firstChild.setValue(j, secondChild.getValue(j)); secondChild.setValue(j, temp); } }}return Arrays.asList(firstChild, secondChild);

}@Overridepublic int getNbCrossoverPoints() {

return this.nbCrossoverPoints;}...

}

@GeneticIO DevoxxUK 2015 #DevoxxUK

Mutation Operator

public class PhilaeMutationOperator implements MutationOperator{...@Overridepublic IndividualBean mutate(IndividualBean individual, Random rng) {

individual.setValue("direction", mutateDirection(individual.getValue("direction"),

rng));return individual;

}private String mutateDirection(String direction, Random rng) {

return ...;}

}

@GeneticIO DevoxxUK 2015 #DevoxxUK

Job Configuration

@GeneticIO DevoxxUK 2015 #DevoxxUK

Fitness Score

@GeneticIO DevoxxUK 2015 #DevoxxUK

Custom Candidate Display

@GeneticIO DevoxxUK 2015 #DevoxxUK

Features• Embedded Secure User Account Management

• Data locality on most popular distributed databases (MongoDB, Cassandra)

• Parallelized Fitness Function Computation with Spark, JPPF or multithreading

• Fully Distributed Job Processing with JPPF

• Customisable candidates representation with D3.js SVG shapes

• Ability to replay jobs for in-depth algorithm evolution diagnosis

• Multilanguage (Java, Scala…), multiplatform (Windows, Unix, Mac)

• Open source!

@GeneticIO DevoxxUK 2015 #DevoxxUK

Demo!

@GeneticIO DevoxxUK 2015 #DevoxxUK

Questions?