Presentation F Sharp

From the math to an F# interfaceImplementing the D.E. algorithm

A numerical optimization library in F#

N. El Bachir1

1FinBoost Limited

29 April 2010

N. El Bachir F# Solvers


Contents

1 From the math to an F# interfaceThe mathematical problemMapping the problem into an F# interface

2 Implementing the D.E. algorithmDifferential Evolution algorithmF# implementation



The mathematical problemMapping the problem into an F# interface

Optimization under constraints

Find x∗ ∈ A sucht that

x∗ = argminx

f (x), (1)

s.t. g(x) (2)

Objective function f defined on domain A (A⊆ Rn) :

f : A→ R (3)

Constraints g(x) an arbitrary Boolean combination of :

equations g(x) = 0,weak inequalities g(x)≥ 0, andweak inequalities g(x)> 0.




Some examples

Very simple : f (x) = x2, A = R and no constraint. Solution :x∗ = 0Another example : f (x ,y) = x− y ,A = R2 s.t.−3x2+2xy − y2 ≥−1. Solution : x∗ = 0, y∗ = 1 andf (0,1) =−1.Third example : f (x ,y) = x2+(y − 1

2)2, A = R2 s.t. y ≥ 0 and

y ≥ x +1.Solution : x∗ =−0.25, y∗ = 0.75 andf (−0.25,0.75) = 0.125.We are interested in numerical algorithms to solve morecomplex problems.




Numerical optimization

Iterative procedure :start with initial guess(es) x0,keep updating x j → x j+1 = F (x j) according to sometransformation rule F ,stop when some criteria are met

# of iterations,convergence in x ,convergence in objective function.

To handle constraints :Transform into unconstrained optimization

drop non-active inequalities.search value of the variable that dominates the constraint.Lagrange multipliers.use of merit/penalty functions.Feasible directions method.

Repairing/rejecting unfeasible solutions.




Solve

let Solve (problem) (strategy) (stopWhen): Solution =let rec helper (pblm) (stp): Solution =

let solution = strategy.oneStepSolve problem stlet isFinished,stopper =

solution.stop.Finished solutionmatch isFinished with| true -> solution| _ -> let solution =

{best=solution.bestbestValue=

solution.bestValuestop = stopper

}helper pblm solution.stop

helper problem stop




Main abstractions

type Problem =abstract Domain : Domain with getabstract Objective : RnOnDomain -> realabstract Constraint : Constraint with get

and Constraint =abstract isSatisfied : RnOnDomain -> bool

and Strategy =abstract oneStepSolve :

Problem -> StopWhen -> Solutionand StopWhen =

abstract Finished : Solution -> bool*StopWhenand Solution =

{best : RnOnDomainbestValue : realstop : StopWhen

}




Domains on Rn

type RnOnDomain =val rn : Rnval dim: intnew(x:Rn,domain:Domain) =

let rn =if domain.contains(x) then xelse failwith ("Not in the domain")

let dim = match x with| Real1 x -> 1| RealN x -> Array.length x

{rn = rndim = dim

}




More on domains

type Domain =val lowBound : BoundVal[]val upBound : BoundVal[]new(bound:Bounds)=

let low,up =let low_,up_= match bound with| Real theBound ->

[|theBound.lower|],[|theBound.upper|]| Realn theBound ->

theBound.lower,theBound.upperif low_.Length = up_.Length then low_,up_else failwith

("Invalid domain: " +"# of upper bounds != # of lower bounds.")

{lowBound = lowupBound = up

}N. El Bachir F# Solvers



More on domains

member this.contains(x) : bool =let X = match x with

| Real1 x -> [| x |]| RealN x -> x

if not( X.Length = this.lowBound.Length )then failwith

("Input has different dimensions than domain")else

let above_lower =[| for k in 0..(this.lowBound.Length - 1) -> let z = this.lowBound.[k]

let x = X.[k]match z with| Inclusive y -> x >= y| Exclusive y -> x > y |] //let below_upper = ...

Array.forall2(fun x y -> x && y) above_lower below_upper

member this.newRn(x:Rn) = new RnOnDomain(x,this)




Reals and intervals

type BoundVal =| Inclusive of float| Exclusive of float

type Range = {lower : BoundValupper : BoundVal

}type Range_nd = {

lower : BoundVal[]upper : BoundVal[]

}type Bounds =

| Real of Range| Realn of Range_nd

type Rn =| Real1 of float| RealN of float[]




Simple example

let bisectPbm ={ new Problem with

member this.Domain = domain1Dmember this.Objective (x:RnOnDomain)

: float= objFuncmember this.Constraint = (

new PosConst domain1D):> Constraint

}let solution = (

Solve (bisectProblem) (new BisectionMethod()) oneStep

).best.Rn



Differential Evolution algorithmF# implementation

Quick intro to D.E.

Stochastic global optimization approach which mimicksevolutionary concepts.Initialization consists in picking an initial population randomlyUpdating procedure (generation evolution) given by

Mutation,Recombination,Selection.




Initial population

D parameters. Initial population of size N, will evolve in eachgeneration G with selection

x i ,G = [x1,i ,G , . . . ,xD,i ,G ] (4)

Bounds for each parameter :

xLj ≤ xj ,i ,1 ≤ xU

j (5)

Initial population randomly selected on the intervals[xLj , x

Uj

](6)




A mutation strategy

For each member i of the population, a mutant is generated byrandomly picking distinct indices k1, k2, and k3 and combiningthem. For example, using a mutation factor F ∈ [0, 2] :

v i ,G+1 = xk1,G +F ×(xk2,G −xk3,G

)(7)

Recombination step will then create a trial member from amutant and a member from previous generation.




Recombination/Crossover

Given a crossover probability CR,randj ,i v U[0, 1] for i = 1, . . . , N and j = 1, . . . , DV a random integer from 1, . . . , DTrial members for the next generation given by

uj ,i ,G+1 =

{v j ,i ,G+1 if randj ,i ≤ CR or j = Vx j ,i ,G+1 otherwise

(8)




Selection of the fittest

Each trial member is compared to its previous generation andthe fittest is selected for each pair (x i ,G , u i ,G+1) :

x i ,G+1 =

{u i ,G+1 if f (u i ,G+1)≤ f (x i ,G )

x i ,G otherwise(9)

The process of mutation, recombination and selectioncontinues until either a maximum number of iterations isreached or some convergence criteria is attained.




Some snippets

Initialization of the population :

initpop = [| for j in 0 .. N-1 -> randomize domain |]// randomize checks the bounds for each dim// returns an array of rands, 1 per dim within the bounds

Mutation and crossover :

for n = 0 to N-1 dofor d = 0 to D-1 do

let ui=if unitRand() < CR then

bm.[n].[d] + F*(pm1.[n].[d] - pm2.[n].[d])else pop.[n].[d]

trial.[n].[d] <- ui




More snippets

Convergence criteria :

type MaxItOrConv (iter:int, tol:float) =interface StopWhen with

member this.Finished solution =let run_iter =

if iter = 0 then trueelse false

let distance = solution.bestValue < tolrun_iter || distance, solution.stop


Technology

Presentation F Sharp