Designing Games for Distributed Optimization Na Li and Jason R. Marden IEEE Journal of Selected...

Designing Games for Distributed Optimization

Na Li and Jason R. Marden

IEEE Journal of Selected Topics in Signal Processing, Vol. 7, No. 2, pp. 230-242, 2013

Presenter: Seyyed Shaho Alaviani

Designing Games for Distributed OptimizationNa Li and Jason R. MardenIEEE Journal of Selected Topics in Signal Processing, Vol. 7, No. 2, pp. 230-242, 2013

Presenter: Seyyed Shaho Alaviani

Introduction -advantages of game theory

Problem Formulation and Preliminaries - potential games -state based potential games -stationary state Nash equilibrium

Main Results - state based game design -analytical properties of designed game -learning algorithm

Numerical Examples

Conclusions

Network-Consensus-Rendezvous-Formation-Schooling-Flocking

All: special cases of distributed optimization

Game Theory: a powerful tool for the design and control of multi agent systems

Using game theory requires two steps:

1- modelling the agent as self-interested decision maker in a game theoretical environment: defining a set of choices and a local objective function for each decision maker

2- specifying a distributed learning algorithm that enables the agents to reach a Nash equilibrium of the designed game

Introduction

Core advantage of game theory:

It provides a hierarchical decomposition between

the distribution and optimization problem (game design)

and

the specific local decision rules (distributed learning algorithm)

Example: Lagrangian

The goal of this paper:

To establish a methodology for the design of local agent objective functions that leads to desirable system-wide behavior

Connected and disconnected graphs Directed and undirected graphs

connected disconnected directed

undirected

Graph

Consider a multi-agent of agents,

set of decisions, nonempty convex subset of real numbers

Optimization problem:

s.t.

where is a convex function, andthe graph is undirected and connected

Problem Formulation and Preliminaries

Physics:

Main properties of potential games:

1- a PSNE is guaranteed to exist

2- there are several distributed learning algorithms with proven asymptotic guarantees

3- learning PSNE in potential games is robust: heterogeneous clock rates and informational delays are not problematic

Stochastic games( L. S. Shapley, 1953):

In a stochastic game the play proceeds by steps from position to position, according to transition probabilities controlled jointly by two players.

State Based Potential Games(J. Marden, 2012):

A simplification of stochastic games that represents and extension to strategic form games where an underlying state space is introduced to the game theoretic environment

State Based Game Design:The goal is to establish a state based game formulation for our distributed optimization problem that satisfies the following properties:

Main Results

A State Based Game Design for Distributed Optimization:

- State Space

- Action sets

- State dynamics

- Invariance associated with state dynamics

- Agent cost functions

State Space:

Action sets:

An action for agent I is defined as a tuple

indicates a change in the agent value

indicates a change in the agent’s estimation term

State Dynamics:

For a state and an action , the ensuing state is given by

Invariance associated with state dynamics:

Let be the initial values of the agents

Define the initial estimation terms to satisfy

Then for all

Agent cost functions:

Analytical Properties of Designed Game

Theorem 2 shows that the designed game is a state based potential game.

Theorem 2: The state based game is a state based potential game with potential function

and represents the ensuing state.

Theorem 3: Let G be the state based game. Suppose that is a differentiable convex function, the communication graph is connected and undirected, and at least one of the following conditions is satisfied:

Theorem 3 shows that all equilibria of the designed game are solutions to the optimization problem.

Question:

Could the results in Theorem 2 and 3 have been attained using framework of strategic form games?

impossible

Learning Algorithm

We prove that the learning algorithm gradient play converges to a stationary state NE.

Assumptions:

Theorem 4: Let G be a state based potential game with a potential function that satisfies the assumption. If the step size for all , then the state action pair of the gradient play

asymptotically converges to a stationary state NE.

Example 1:

Consider the following function to be minimized

Numerical Examples

Example 2: Distributed Routing Problem

source destination

m routes

Application: the Internet

Amount traffic

Percentage of traffic that agent i designates to route r

For each route r, there is an associated congestion function that reflects the cost of using the route as a function of the amount of traffic on that route.

Then total congestion in the network will be

R=5

N=10

Communication graph

𝛼=900

Conclusions:

- This work presents an approach to distributed optimization using the framework of state based potential games.

- We provide a systematic methodology for localizing the agents’ objective functions while ensuing that the resulting equilibria are optimal with regards to the system level objective function.

- It is proved that the learning algorithm gradient play guarantees convergence to a stationary state NE in any state based potential game

- Robustness of the approach

MANY THANKS

FOR

YOUR ATTENTION