Designing Games for Distributed Optimization
Na Li and Jason R. Marden
IEEE Journal of Selected Topics in Signal Processing, Vol. 7, No. 2, pp. 230-242, 2013
Presenter: Seyyed Shaho Alaviani
Designing Games for Distributed OptimizationNa Li and Jason R. MardenIEEE Journal of Selected Topics in Signal Processing, Vol. 7, No. 2, pp. 230-242, 2013
Presenter: Seyyed Shaho Alaviani
Introduction -advantages of game theory
Problem Formulation and Preliminaries - potential games -state based potential games -stationary state Nash equilibrium
Main Results - state based game design -analytical properties of designed game -learning algorithm
Numerical Examples
Conclusions
Network-Consensus-Rendezvous-Formation-Schooling-Flocking
All: special cases of distributed optimization
Game Theory: a powerful tool for the design and control of multi agent systems
Using game theory requires two steps:
1- modelling the agent as self-interested decision maker in a game theoretical environment: defining a set of choices and a local objective function for each decision maker
2- specifying a distributed learning algorithm that enables the agents to reach a Nash equilibrium of the designed game
Introduction
Core advantage of game theory:
It provides a hierarchical decomposition between
the distribution and optimization problem (game design)
and
the specific local decision rules (distributed learning algorithm)
Example: Lagrangian
The goal of this paper:
To establish a methodology for the design of local agent objective functions that leads to desirable system-wide behavior
Connected and disconnected graphs Directed and undirected graphs
connected disconnected directed
undirected
Graph
Consider a multi-agent of agents,
set of decisions, nonempty convex subset of real numbers
Optimization problem:
s.t.
where is a convex function, andthe graph is undirected and connected
Problem Formulation and Preliminaries
Physics:
Main properties of potential games:
1- a PSNE is guaranteed to exist
2- there are several distributed learning algorithms with proven asymptotic guarantees
3- learning PSNE in potential games is robust: heterogeneous clock rates and informational delays are not problematic
Stochastic games( L. S. Shapley, 1953):
In a stochastic game the play proceeds by steps from position to position, according to transition probabilities controlled jointly by two players.
State Based Potential Games(J. Marden, 2012):
A simplification of stochastic games that represents and extension to strategic form games where an underlying state space is introduced to the game theoretic environment
State Based Game Design:The goal is to establish a state based game formulation for our distributed optimization problem that satisfies the following properties:
Main Results
A State Based Game Design for Distributed Optimization:
- State Space
- Action sets
- State dynamics
- Invariance associated with state dynamics
- Agent cost functions
State Space:
Action sets:
An action for agent I is defined as a tuple
indicates a change in the agent value
indicates a change in the agent’s estimation term
State Dynamics:
For a state and an action , the ensuing state is given by
Invariance associated with state dynamics:
Let be the initial values of the agents
Define the initial estimation terms to satisfy
Then for all
Agent cost functions:
Analytical Properties of Designed Game
Theorem 2 shows that the designed game is a state based potential game.
Theorem 2: The state based game is a state based potential game with potential function
and represents the ensuing state.
Theorem 3: Let G be the state based game. Suppose that is a differentiable convex function, the communication graph is connected and undirected, and at least one of the following conditions is satisfied:
Theorem 3 shows that all equilibria of the designed game are solutions to the optimization problem.
Question:
Could the results in Theorem 2 and 3 have been attained using framework of strategic form games?
impossible
Learning Algorithm
We prove that the learning algorithm gradient play converges to a stationary state NE.
Assumptions:
Theorem 4: Let G be a state based potential game with a potential function that satisfies the assumption. If the step size for all , then the state action pair of the gradient play
asymptotically converges to a stationary state NE.
Example 1:
Consider the following function to be minimized
Numerical Examples
Example 2: Distributed Routing Problem
source destination
m routes
Application: the Internet
Amount traffic
Percentage of traffic that agent i designates to route r
For each route r, there is an associated congestion function that reflects the cost of using the route as a function of the amount of traffic on that route.
Then total congestion in the network will be
R=5
N=10
Communication graph
𝛼=900
Conclusions:
- This work presents an approach to distributed optimization using the framework of state based potential games.
- We provide a systematic methodology for localizing the agents’ objective functions while ensuing that the resulting equilibria are optimal with regards to the system level objective function.
- It is proved that the learning algorithm gradient play guarantees convergence to a stationary state NE in any state based potential game
- Robustness of the approach
MANY THANKS
FOR
YOUR ATTENTION