Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Distributed Parallel Process Particle SwarmOptimization on Fixed Charge Network

Flow Problems

Corey Clark1 Charles Nicholson2

1Game Theory Labs, Dallas, TXcclark@gametheorylabs.com

2University of Oklahoma, Industrial and Systems Engineering, Norman, OKcnicholson@ou.edu

INFORMS Annual Meeting, 2013

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Outline

1 IntroductionThe ProblemThe Model

2 Problem ApproachesOptimal SearchHeuristic Search

3 Dynamically Distributed BPSO ApproachAlgorithmArchitecture

4 Performance Results and DemonstrationPerformance ResultsDemonstration

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

The Problem

Problem Motivation: Cash Management2

National banks managevaults that store cashVaults have excess ordeficit (current / forecast)Routing cash incursfixed and variable costsModeled as Time-spacefixed-charge networkflow problem1

1J. Kennington and C. Nicholson. “The Uncapacitated Time-Space Fixed-ChargeNetwork Flow Problem: An Empirical Investigation of Procedures for Arc CapacityAssignment”. In: INFORMS Journal on Computing 22 (2010), pp. 326–337.

2M. Frost, J. Kennington, and A. Madhavan. “Optimizing cash management for largescale bank operations”. In: International Journal of Operations Research andInformation Systems 1 (2010), pp. 17–31.

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

The Model

Time-Space Fixed-Charge Network Flow Model

Graph

N is set of n spatial nodes; T is the set of t time periods, and N̄is the set of node-time pairsA is the set of arcs: (i , r , j , s) where (i , r) ∈ N̄, (j , s) ∈ N̄

Variablesxirjs is flow on arc (i , r , j , s) ∈ A; yirjs is related binary variable

ParametersMirjs is an implied artificial arc capacity used in B&B modelingcirjs and firjs are variable and fixed costs for arc (i , r , j , s)Rir are the requirements at node (i , r) ∈ N̄.

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

The Model

Time-Space Fixed-Charge Network Flow Model

Graph

N is set of n spatial nodes; T is the set of t time periods, and N̄is the set of node-time pairsA is the set of arcs: (i , r , j , s) where (i , r) ∈ N̄, (j , s) ∈ N̄

Variablesxirjs is flow on arc (i , r , j , s) ∈ A; yirjs is related binary variable

ParametersMirjs is an implied artificial arc capacity used in B&B modelingcirjs and firjs are variable and fixed costs for arc (i , r , j , s)Rir are the requirements at node (i , r) ∈ N̄.

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

The Model

Time-Space Fixed-Charge Network Flow Model

Graph

N is set of n spatial nodes; T is the set of t time periods, and N̄is the set of node-time pairsA is the set of arcs: (i , r , j , s) where (i , r) ∈ N̄, (j , s) ∈ N̄

Variablesxirjs is flow on arc (i , r , j , s) ∈ A; yirjs is related binary variable

ParametersMirjs is an implied artificial arc capacity used in B&B modelingcirjs and firjs are variable and fixed costs for arc (i , r , j , s)Rir are the requirements at node (i , r) ∈ N̄.

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

The Model

TSFC Problem Formulation

Given a directed graph G = (N̄,A) the time-space fixed-chargenetwork flow model is formally stated as follows:

TSFC Problem

min∑

(i,r ,j,s)∈A

(cirjsxirjs + firjsyirjs) (1)

s.t.∑

(i,r ,j,s)∈A

xirjs −∑

(j,s,i,r)∈A

xjsir = Rir ∀(i , r) ∈ N̄ (2)

0 ≤ xirjs ≤ Mirjsyirjs ∀(i , r , j , s) ∈ A (3)yirjs ∈ {0,1} ∀(i , r , j , s) ∈ A (4)

This problem is NP-hard.

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Optimal Search

Optimization Approach

Branch-and-Bound for the TSFC problem1 Problem is solved with relaxed binary constraints2 Choose an arc (i , r , j , s) ∈ A for branching

create new subproblem with yirjs = 1create new subproblem with yirjs = 0

3 Solve the relaxed sub-problems (linear programs)4 Use results to update bounds, determine optimality, fathom

nodes, and continued branching5 Go to step 2

B&B performs a complete (implicit) search among all possiblenetwork designs (branches) to find an optimal solution.

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Optimal Search

Search Space

The set of all feasible solutionsis called a search spaceEach point in the spacerepresents one feasiblesolutionEvery point has an associatedfitness valueThe set of solutions and theirobjective values form locationsand elevation in the searchspace landscapeThe search space can belarge and complex

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Heuristic Search

Heuristic Approach

LiteratureTabu search (Glover 1990)

Fixed-Charge Transportation Problem (Sun et al. 1998)Genetic Algorithms (Holland 1975)

Non-linear Transportation Problem (Sheng et al. 2006)Fixed-Charge Network Flow (Duhamel 2010)Network Designs (Gen and Chang 2003)

Particle Swarm Optimization (PSO)Real-valued PSO (Kennedy and Eberhart 1995)Binary PSO (Kennedy and Eberhart 1997)Many applications and enhancements since then, e.g.Parsopoulos and Vrahatis 2007, Yin et. al 2010

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Algorithm

Binary Particle Swarm Optimization

Particle position and velocityXi = (xi1, xi2, . . . , xik ) xij ∈ {0,1} for j = 1,2, . . . , kVi = (vi1, vi2, . . . , vik ) vii ∈ [−vmax , vmax ] for j = 1,2, . . . , k

The position Xi represents a solution, and has anassociated fitness valueA swarm of n particles fly through the k -dimensionalsolution space – updating their positions based on avelocity functionVelocity is a function of the historical best solution foundindividually by a particle and globally by the swarmThe velocity is converted into a probability that is used tomodify the particle position

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Algorithm

Binary Particle Swarm Optimization

Particle position and velocityXi = (xi1, xi2, . . . , xik ) xij ∈ {0,1} for j = 1,2, . . . , kVi = (vi1, vi2, . . . , vik ) vii ∈ [−vmax , vmax ] for j = 1,2, . . . , k

The position Xi represents a solution, and has anassociated fitness value

A swarm of n particles fly through the k -dimensionalsolution space – updating their positions based on avelocity functionVelocity is a function of the historical best solution foundindividually by a particle and globally by the swarmThe velocity is converted into a probability that is used tomodify the particle position

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Algorithm

Binary Particle Swarm Optimization

Particle position and velocityXi = (xi1, xi2, . . . , xik ) xij ∈ {0,1} for j = 1,2, . . . , kVi = (vi1, vi2, . . . , vik ) vii ∈ [−vmax , vmax ] for j = 1,2, . . . , k

The position Xi represents a solution, and has anassociated fitness valueA swarm of n particles fly through the k -dimensionalsolution space – updating their positions based on avelocity function

Velocity is a function of the historical best solution foundindividually by a particle and globally by the swarmThe velocity is converted into a probability that is used tomodify the particle position

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Algorithm

Binary Particle Swarm Optimization

Particle position and velocityXi = (xi1, xi2, . . . , xik ) xij ∈ {0,1} for j = 1,2, . . . , kVi = (vi1, vi2, . . . , vik ) vii ∈ [−vmax , vmax ] for j = 1,2, . . . , k

The position Xi represents a solution, and has anassociated fitness valueA swarm of n particles fly through the k -dimensionalsolution space – updating their positions based on avelocity functionVelocity is a function of the historical best solution foundindividually by a particle and globally by the swarm

The velocity is converted into a probability that is used tomodify the particle position

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Algorithm

Binary Particle Swarm Optimization

Particle position and velocityXi = (xi1, xi2, . . . , xik ) xij ∈ {0,1} for j = 1,2, . . . , kVi = (vi1, vi2, . . . , vik ) vii ∈ [−vmax , vmax ] for j = 1,2, . . . , k

The position Xi represents a solution, and has anassociated fitness valueA swarm of n particles fly through the k -dimensionalsolution space – updating their positions based on avelocity functionVelocity is a function of the historical best solution foundindividually by a particle and globally by the swarmThe velocity is converted into a probability that is used tomodify the particle position

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Algorithm

Binary Particle Swarm Optimization Updates

Let Pi denote the best position visited by particle iLet Pg denote the best position found by the swarm

The velocity function at iteration t + 1 is:

V t+1i = wV t

i + C1r1(Pi − X t

i)

+ C2r2(Pg − X t

i)

where w is an inertia factor that changes over time; C1,C2 areconstants; r1, r2 are random values uniform on [0,1]

The transfer function that converts velocities to probabilities

T (vij) =1

1 + evij

is used to update each bit the position according to:

xij =

{1, if U(0,1) < T (vij)

0, otherwise

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Algorithm

Binary PSO and the FCNF Problem

In our application of Binary PSO tothe Fixed-Charge Network FlowProblem, each particle positionrepresents a network design.

Each position is a unique, purenetwork problem which is solved todetermine the minimum cost of theparticular design.

The individual designs are entirelyindependent of each other, whichmakes this easy to parallelize.

X1 :

X2 :

X3 :

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Architecture

Distributed Accelerated Analytics Platform

Distributed Accelerated Analytics Platform (DAAP)DAAP enables dynamic creation of a heterogeneous clusternetwork using HTML5 technologies that provide multi-threadedexecution and low latency connections.

GLPK and GLPK.jsThe GNU Linear Programming Kit (GLPK) package solveslarge-scale LP and MIP problems.GLPK.js: JavaScript interfaces to GLPK which allow GLPK tobe called from webpages on either the client side or the serverside.3

Any web-enabled device can become part of a DAAP clusternetwork and contribute computing power to solve LP problems.

3H. Gourvest. GLPK.js. URL: https://github.com/hgourvest/glpk.js.

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Architecture

Distributed Accelerated Analytics Platform

Distributed Accelerated Analytics Platform (DAAP)DAAP enables dynamic creation of a heterogeneous clusternetwork using HTML5 technologies that provide multi-threadedexecution and low latency connections.

GLPK and GLPK.jsThe GNU Linear Programming Kit (GLPK) package solveslarge-scale LP and MIP problems.GLPK.js: JavaScript interfaces to GLPK which allow GLPK tobe called from webpages on either the client side or the serverside.3

Any web-enabled device can become part of a DAAP clusternetwork and contribute computing power to solve LP problems.

3Gourvest, GLPK.js.

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Performance Results

Empirical Testing: Problem Characteristics

We tested our approach on three different sized TSFCProblems. Feasible instances were randomly generated andthe variable and fixed cost ranges were selected such as tocreate difficult instances (Kennington and Nicholson 2010).

Problems

Variable Costs: U(0,10)Fixed Costs: U(20000,60000)

Size Nodes Arcs5n6p 30 24510n21p 210 3,89020n30p 600 23,000

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Performance Results

Empirical Testing: Approaches

Three approaches were tested:Optimization: GLPK.js with default MIP techniques

Serial BPSO-FCNFDistributed BPSO-FCNF

Devices used in the DAAP cluster include: MacBookPro,Surface Pro, iPhone, Windows laptop, iPad, Nexus 7tablet

A time limit was set at 10 minutes and best objective valuesand number of network problems solved are reported.

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Performance Results

Empirical Testing: Approaches

Three approaches were tested:Optimization: GLPK.js with default MIP techniquesSerial BPSO-FCNF

Distributed BPSO-FCNFDevices used in the DAAP cluster include: MacBookPro,Surface Pro, iPhone, Windows laptop, iPad, Nexus 7tablet

A time limit was set at 10 minutes and best objective valuesand number of network problems solved are reported.

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Performance Results

Empirical Testing: Approaches

Three approaches were tested:Optimization: GLPK.js with default MIP techniquesSerial BPSO-FCNFDistributed BPSO-FCNF

Devices used in the DAAP cluster include: MacBookPro,Surface Pro, iPhone, Windows laptop, iPad, Nexus 7tablet

A time limit was set at 10 minutes and best objective valuesand number of network problems solved are reported.

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Performance Results

Empirical Testing: Approaches

Three approaches were tested:Optimization: GLPK.js with default MIP techniquesSerial BPSO-FCNFDistributed BPSO-FCNF

Devices used in the DAAP cluster include: MacBookPro,Surface Pro, iPhone, Windows laptop, iPad, Nexus 7tablet

A time limit was set at 10 minutes and best objective valuesand number of network problems solved are reported.

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Performance Results

Empirical Testing: Approaches

Three approaches were tested:Optimization: GLPK.js with default MIP techniquesSerial BPSO-FCNFDistributed BPSO-FCNF

Devices used in the DAAP cluster include: MacBookPro,Surface Pro, iPhone, Windows laptop, iPad, Nexus 7tablet

A time limit was set at 10 minutes and best objective valuesand number of network problems solved are reported.

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Performance Results

Empirical Testing: Performance

Computational Time: 10 minutes

GLPK Serial Distributed Serial vs.Problem Opt BPSO-FCNF BPSO-FCNF Distributed

5n6p1.3M 937K 907K (37K)285K 300K 1.3M (4.3x)

10n21pN/A 7.36M 7.21M (150K)5.2K 30K 68K (2.3x)

20n30pN/A 21.16M 20.97M (190k)3.2K 1.2K 3.3K (2.8x)

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Demonstration

I hope this works...

pso.gametheorylabs.com1 Navigate to this website2 Register your device as a Processing Node3 Watch for the green “loaded” and “running” indicators

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Summary

Binary PSO (with modifications) is an effective option fordifficult Fixed-Charge Network Flow problems.

Binary PSO for FCNF are naturally independent and thuseasily parallelized.Using the DAAP technology, it is easy to create dynamiccluster computing to address difficult MIP and problemswith such techniques.Questions?

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Summary

Binary PSO (with modifications) is an effective option fordifficult Fixed-Charge Network Flow problems.Binary PSO for FCNF are naturally independent and thuseasily parallelized.

Using the DAAP technology, it is easy to create dynamiccluster computing to address difficult MIP and problemswith such techniques.Questions?

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Summary

Binary PSO (with modifications) is an effective option fordifficult Fixed-Charge Network Flow problems.Binary PSO for FCNF are naturally independent and thuseasily parallelized.Using the DAAP technology, it is easy to create dynamiccluster computing to address difficult MIP and problemswith such techniques.

Questions?

Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary

Summary

Binary PSO (with modifications) is an effective option fordifficult Fixed-Charge Network Flow problems.Binary PSO for FCNF are naturally independent and thuseasily parallelized.Using the DAAP technology, it is easy to create dynamiccluster computing to address difficult MIP and problemswith such techniques.Questions?