Embed Size (px)
Citation preview
1
Modeling, Analysis, and Control of Modeling, Analysis, and Control of MultiMulti--agent Systems agent Systems
Jinhu Lü
Academy of Mathematics and Systems Science Chinese Academy of Sciences
China-EU Summer School, Shanghai, 12 August 2010
2
With Thanks ToUSST and USTC, ChinaProf. Bing-Hong WangCity University of Hong KongProf. Guanrong ChenRMIT University, AustraliaProf. Xinghuo YuPrinceton University, USAProf. Simon Levin and Prof. Iain CouzinUniversity of Virginia, USAProf. Zongli LinAMSS, Chinese Academy of SciencesYao Chen
3
Outline
Introduction
Modeling, Analysis, Control of MAS
Some Advances in Consensus of MAS
Conclusions
4
Part I: Introduction
Some Examples
What is Multi-Agent Systems (MAS)?
Main Characteristics of MAS
5
Examples: Bird Flock
6
Examples: Fish Flock
7
Examples: Rotating Ants Mill
8
Examples: Bacteria Group
9
Examples: Social Systems
10
The problem: Maintaining a formation in 2D or 3D
Examples: Formation Control
Autonomous agent
11
Examples: Group Robots
12
What is Multi-Agent Systems (MAS)?
AgentsInsect, bird, fish, people, robot, …node, individual, particle, …Local Rules
13
Main Characteristics of MAS
Autonomous/Self-Driven Agents
Distributed Region
Local Interactions
Dynamic Neighbors
Various Connections
Conformable Behaviors
14
Outline
Introduction
Modeling, Analysis, Control of MAS
Some Advances in Consensus of MAS
Conclusions
15
Part II: Modeling, Analysis, and Control of MAS
Modeling—Vicsek, Boid, Couzin-Levin, Complex Dynamical Networks
Analysis—Consensus, Convergence, Adaptation, Decision-Making, …
Control—Leader-Follower, Pinning Control, Formation Control,
Cooperation, Intervention, …
16
Emergent Behaviors
Many Individuals
Many Interactions
Decentralization Simple Local Rules ?
Collective Behaviors: Collective Behaviors: Modeling, Analysis and ControlModeling, Analysis and Control
A Unifying Framework of MAS
17
Several Representative Models:Boids Flocking Model (1987)
Reynolds, Flocks, herd, and schools: A distributed behavioralmodel, Computer Graphics, 1987, 21(4): 15-24.
Three Basic Local Rules: (The First Model)Alignment: Steer to move toward the average heading of
local flockmatesSeparation: Steer to avoid crowding local flockmatesCohesion: Steer to move toward the average position of
local flockmates
18
Several Representative Models:Vicsek Particles Model (1995)
Vicsek, et al., Novel type of phase transition in a system of seVicsek, et al., Novel type of phase transition in a system of selflf--driven driven particles, Phys. Rev. particles, Phys. Rev. LettLett., 1995, 75 (6): 1226.., 1995, 75 (6): 1226.
Position：
Heading：
One Basic Local Rule: Alignment (The Simplest Model)
19
Several Representative Models:Vicsek Particles Model (1995)
Vicsek, et al., Novel type of phase transition in a system of seVicsek, et al., Novel type of phase transition in a system of selflf--driven driven particles, Phys. Rev. particles, Phys. Rev. LettLett., 1995, 75 (6): 1226.., 1995, 75 (6): 1226.
(a) Initial: Random positions/velocities
(b) Low density/noise: grouped, random
(c) High density/noise: correlated, random
(d) High density / Low noiseordered motion
20
Several Representative Models:Couzin-Levin Model (2005)
Updating rules: (Information)
Couzin, I. D., Krause, J., Franks, N. R. & Levin, S. M. Effective leadership and decision-making in animal groups on the Move, Nature, 2005, 433: 513-516.
Local Rules: Separation, Cohesion, Alignment
21
Analysis of MAS: Main Approaches
Numerical Simulations
——Simulation Platform, Mathematical or Computer Model
Theoretical Analysis——Lyapunov Function, Eigenvalue Computation, Convex
Analysis, Stochastic Approximation, Graph Theory, …
Experimental Observation——Fish, Locust, Bird, Ants, …
22
Numerical Simulations
Clapping Synchronization
NatureNature
Panic Escape
LearningLearning
AdaptationAdaptation
CooperationCooperation
CompetitionCompetition
…………
23
Simulation Platform of Collective Behaviors of MASSimulation Platform of Collective Behaviors of MAS
24
Simplification of Simplification of Vicsek modelVicsek model by linearization:by linearization:
A. A. JadbabieJadbabie, J. Lin, and A.S. Morse, Coordination of groups , J. Lin, and A.S. Morse, Coordination of groups ofmobileofmobile autonomous agents autonomous agents using nearest neighbor rules, using nearest neighbor rules, IEEETransIEEETrans. Automat. Contr., 2003, 48(6): 988. Automat. Contr., 2003, 48(6): 988--10011001
Analysis of this model is based on Analysis of this model is based on WolfowitzWolfowitz Theorem:Theorem:
Given a set of finite number of SIA matrices, if any finiteGiven a set of finite number of SIA matrices, if any finiteproducts generated from this set is SIA, then any infiniteproducts generated from this set is SIA, then any infiniteproducts generated from this set is convergentproducts generated from this set is convergent
J. J. WolfowitzWolfowitz, Products of indecomposable, , Products of indecomposable, aperiodicaperiodic, stochastic matrices, Proc. Amer. , stochastic matrices, Proc. Amer. Math. Soc, 1963, 15: 733Math. Soc, 1963, 15: 733--737.737.
( )
1( 1) ( )| ( ) |
i
i jj N ti
t tN t
θ θ∈
+ = ∑
Theoretical Analysis
25
Continuous-Time vs. Discrete-Time
Fixed TopologyFixed TopologyContinuousContinuous--Time:Time:Lyapunov Function, Lyapunov Function, EigenvalueEigenvalue ComputationComputationDiscreteDiscrete--Time: Time: Lyapunov Function, Lyapunov Function, EigenvalueEigenvalue ComputationComputation
Switching topologySwitching topologyContinuousContinuous--Time:Time:Lyapunov FunctionLyapunov FunctionDiscreteDiscrete--Time: [Time: [Nonexistence of quadratic Lyapunov functionNonexistence of quadratic Lyapunov function]]
Convex Analysis, Stochastic Approximation, Graph TheoryConvex Analysis, Stochastic Approximation, Graph TheoryA. A. OlshevskyOlshevsky, J. N. , J. N. TsitsiklisTsitsiklis, On the nonexistence of quadratic Lyapunov function for , On the nonexistence of quadratic Lyapunov function for consensus algorithms, consensus algorithms, IEEE IEEE Trans. Automat. Contr., 2008, 53(11): 2642Trans. Automat. Contr., 2008, 53(11): 2642--2645.2645.
26
Experimental Observation
SCIENCESCIENCEFish Migration / MotionFish Migration / Motion
Locust BreedingLocust Breeding
BirdBird Migration
……
27
Control of MAS: Main Approaches
LeaderLeader--Follower ControlFollower ControlCoordinated Control Coordinated Control SSwarming, warming, CConsensus, onsensus, FFlockinglocking, , ……Data Traffic ControlData Traffic ControlSShortesthortest--PPathath, , BBetweennessetweenness, , ……Switch ControlSwitch ControlSwitch Rule, Switch Times, Switch Rule, Switch Times, ……Pinning ControlPinning ControlSelective Scheme, Network Structure, Node Dynamics, Selective Scheme, Network Structure, Node Dynamics, ……InterventionIntervention……
28
Outline
Introduction
Modeling, Analysis, Control of MAS
Some Advances in Consensus of MAS
Conclusions
29
Part III: Some Advances in Consensus of MAS
Estimate the Convergence Rate of MASEstimate the Convergence Rate of MAS
Nonlinear Local RulesNonlinear Local Rules
InformedInformed--Uninformed Local RulesUninformed Local Rules
30
CASE I: Estimate the Convergence Rate of MASEstimate the Convergence Rate of MAS
Y. Chen, D. W. C. Ho, J. Lü, Z. Lin, Convergence rate analysis for discrete-time multi-agent systems with time-varying delays, CCC, 2010.
31
Estimate the Convergence Rate of MASEstimate the Convergence Rate of MAS
Model DescriptionModel Description
AssumptionsAssumptionsA1:A1:A2:A2: There is some such that and for anyThere is some such that and for any ..A3A3: Suppose and , each : Suppose and , each contains acontains a
spanning tree.spanning tree.
Definition of Definition of Convergence RateConvergence Rate
where .
1( 1) ( ) ( ( ))
Ni
i ij j jj
x t a t x t tτ=
+ = −∑
( ) 01
( ) 0, ( ) 1, ( ) 0, inf (0,1).ij
N
ij ij ii a tj
a t a t a t α>=
≥ = > ≥ ∈∑( )i
j t Bτ ≤B ( ) 0ii tτ = t
{ } 0 1[ , )k k kt t t∞= +=∪ 1k kt t T+ − ≤ 1 1 ( ( ))k
k
tt t G A t+ −=∪
1/
(0) 0( )sup limsup(0)
t
ttρ Δ ≠ →∞
⎛ ⎞Δ= ⎜ ⎟Δ⎝ ⎠
, ,( ) max ( ) min ( )i V t B t i i V t B t it x xτ ττ τ∈ − < ≤ ∈ − < ≤Δ = −
32
Theorem:Theorem: Under the assumptions A1Under the assumptions A1--A3, A3, the convergence the convergence raterate for the basic linear MAS satisfyingfor the basic linear MAS satisfying
withwith
Main Idea:Main Idea:Constructing the following sets:Constructing the following sets:
1/(1 )B K Kρ α +≤ −
1 1
2 2
1 1*
** *
* *
1
2
( ) { : ( ) ( 1) (1 ) (0)},
( ) { : ( ) ( 1) (1 ) (0)},
( ) ( ),( ) ( ),
{ : [0, ), , ( ) ( 1)},{ : [0, ), , ( ) ( 1)}.
iB B
i
i
i
S i V M m B M
S i V m M B m
S V SS V S
B i V x m BB i V x M B
μ ξ μ ξ
μ ξ μ ξ
μ μ α α
μ μ α α
μ μμ μ
ξ ξ ξ ξξ ξ ξ ξ
+ − + −
+ − + −
= ∈ > − + −
= ∈ < − + −
= −= −
∈ ∈ ∃ ∈ = −∈ ∈ ∃ ∈ = −
2 2 ( 1)(2 2)K B N T B= − + − + −
Main ResultsMain Results
33
This idea can be illustrated by the following figureThis idea can be illustrated by the following figure
The sets shown in the graph have the following propertyThe sets shown in the graph have the following property* *
* **
*
( 1) ( ), 1( 1) ( ), 1
( ) ( ) , 2 2
S S BS S B
S S B
μ μ μμ μ μ
μ μ φ μ
+ ⊆ ≥ −+ ⊆ ≥ −
= ≥ −∩
Main ResultsMain Results
34
CASE II: Nonlinear Local Rules
Y. Chen, J. Lü, Z. Lin, Consensus of discrete-time multi-agent systems with nonlinear local rules and time-varying delays, CDC, 2009, pp. 7018-7023.
35
( )ix t
( )
1( 1) ( )| ( ) |
i
i jj N ti
x t x tN t ∈
+ = ∑
{1,2,..., }i V n∈ =
Background: Linear Local Rules
( )jx t
x i n2
n1
36
Some Related ReferencesA. Jadbabaie, J. Lin and A. S. Morse, IEEE-TAC, 2003R. Olfati-Saber and R. M. Murry, IEEE TAC, 2004L. Moreau, IEEE TAC, 2005W. Ren and R. W. Beard, IEEE TAC, 2005J. Cortes, S. Martinez, and F.Bullo, IEEE TAC, 2006R. Olfati-Saber, IEEE TAC, 2006Z. Liu and L. Guo, Science in China Ser. F, 2007J. Lin, A. S. Morse, and B. D. O. Anderson, SIAM, 2007F. Cucker and S. Smale, IEEE TAC, 2007M. Cao, A. S. Morse, and B. D. O. Anderson, IEEE TAC, 2008B. Liu and T. Chen, IEEE TNN, 2008B. Liu, W. Lu, and T. Chen, IEEE CDC, 2009W. Yu, G. Chen, M. Cao, J. Lü, 2009……
37
There are few results reported on nonlinear local rules.Vicsek model (PRL, 1995):
Nonlinear
L. Moreau, Stability of multiagent systems with time-dependent communication links, IEEE TAC, 2005, 50(2): 169-182. L. Fang, P.J. Antsaklis, Asynchronous consensus protocols using nonlinear paracontractions theory, IEEE TAC, 2008, 53(10): 2351-2355.
Nonlinear Local Rules
( )
( )
sin( ( ( )))( 1) arctan
cos( ( ( )))
( 1) ( ) cos( ( 1))( 1) ( ) sin( ( 1))
i
i
ij j
j N ti i
j jj N t
i i i
i i i
t tt
t t
x t x t v ty t y t v t
θ τθ
θ τ
θθ
∈
∈
⎛ ⎞−⎜ ⎟
+ = ⎜ ⎟−⎜ ⎟⎝ ⎠
+ = + +
+ = + +
∑∑
38
( )ix t
Nonlinear Local Rules
( )jx t( )ijy t
( ) ( ( ( )))i ij j jy t f x t tτ= −
( )
1( 1) ( ( ))( )
i
ii j
j N ti
x t F y tn t ∈
+ = ∑
39
( )
1( 1) ( ( ( ( ))))( )
i
ii j j
j N ti
x t F f x t tn t
τ∈
+ = −∑
What kinds of functions f, F and G(t) can guarantee the consensus of MAS?
*G(t)=(V, E(t)) is the graph corresponding to the relation among V..
Consensus of MAS: ,i j V∀ ∈lim | ( ) ( ) | 0i jtx t x t
→∞− =
One Open Problem
(1 )
40
Let (A1): f and g are both continuous functions defined on [a, b]
and .(A2): f is monotonous increasing and g is strict monotonous
increasing.(A3): for .(A4): There exists an integer such that is
strongly connected for any .(A5) There exists an integer satisfying for any
.(A6): for .
1g F −=
([ , ]) ([ , ])f a b g a b⊆
( ) ( )f x g x≥ [ , ]x a b∈0Γ > 1 ( )s G t sΓ
= +∪
0t ≥0B >
( ) 0ii tτ = 0t∀ ≥
0 ( )ij t Bτ≤ <
i j≠
Assumptions (1)
41
Assumptions (2)Let
(A4’): is directed, but is undirected and strongly
connected.
(A4’’): There exists some integer when
and there is some with
satisfying .
(A3’): for .( ) ( )f x g x≤ [ , ]x a b∈
( ) lim ( ).i kkG G i≥→∞
∞ = ∪
( )G ∞( )G t
0Γ ≥ ( , ) ( )i j G∈ ∞
( , ) ( )i j G t∈ ⇒ 't t t≤ ≤ +Γ't
( , ) ( ')j i G t∈
(A4’): is directed, but is strongly connected.
( A4’’): There exists some integer such that the in-degree
of each node in equal to its corresponding
out-degree.
( )G t ( )G ∞
0Γ >
1 ( )k sG G k sΓ== +∪
42
Theorem 1: Suppose that (A1)-(A6) hold for a given MAS (V, G(t), (1)). For any given initial states, the states of all agents can reach consensus.
Main Results (1)
43
Theorem 2: Suppose that Assumptions (A1)-(A3), (A5) and (A6) hold for a given MAS (V, G(t), (1)). If there exists a sequence and an integer such that there exists a directed path from to
and for . Then the MAS (V, G(t), (3)) can reach consensus.
{ }kt 0mλ >
( )m ktΩ
( )m ktΩ 10 k k mt t λ+< − < 0k∀ >
{1,2,..., }
{1,2,..., }
( ) { : ( ) min ( )},
( ) { : ( ) max ( )}.
m i jj n
m i jj n
t i m t m t
t i m t m t∈
∈
Ω = =
Ω = =
Main Results (2)
44
( ), ( ), ( }){ i i ix t y t tθ
{ }2 2( ) : ( ( ) ( )) ( ( ) ( )) , 0i i j i jN t j V x t x t y t y t r r= ∈ − + − < >
( )
( )
sin( ( ( )))( 1) arctan
cos( ( ( )))
( 1) ( ) cos( ( 1))( 1) ( ) sin( ( 1))
i
i
ij j
j N ti i
j jj N t
i i i
i i i
t tt
t t
x t x t v ty t y t v t
θ τθ
θ τ
θθ
∈
∈
⎛ ⎞−⎜ ⎟
+ = ⎜ ⎟−⎜ ⎟⎝ ⎠
+ = + +
+ = + +
∑∑
(2)
Application To Vicsek Model
45
Theorem 3: Suppose that Assumptions (A5) and (A6) hold for the Vicsek model with time-varying delays in the updating rules (2). Also, assume that the graph
is connected. If the initial headings , then the headings of agents can reach consensus.
( )G ∞ ( ) ( , )2 2i t π πθ ∈ −
( )
cos ( ( ))( )
cos ( ( ))
( ) tan ( )i
ij j
ij ij j
j N t
i i
t ta t
t t
x t t
θ τθ τ
θ∈
−=
−
=
∑( )
( 1) ( ) ( ( ))i
ii ij j j
j N t
x t a t x t tτ∈
+ = −∑
Application To Vicsek Model
46
Let 1 0 1
2 1 22 2 3
1 4 3 63
x xx x
f x x
x x
+ ≤ ≤⎧⎪ ≤ ≤⎪⎪= + ≤ ≤⎨⎪⎪ + ≤ ≤⎪⎩
1 0 4,( ) 2
2 6 4 6.
x xF x
x x
⎧ ≤ ≤⎪= ⎨⎪ − ≤ ≤⎩
{ : ( ) ( )} { :1 2, 6}aS x f x g x x x x= = = ≤ ≤ =
2B =
Numerical Simulations
The phase portrait of f(x) and F(x)
47
( 1) (1.12,1.52,0.58,1.65,0.17,0.17,0.76,2.06) ,(0) (0.60,0.98,0.16, 2.08,0.00,1.59,0.65, 2.39) .
T
T
xx− =
=
Numerical Simulations (1)
Phase trajectories from the initial states
1 { :1 2}S x x= ≤ ≤
48
Numerical Simulations (2)
Phase trajectories from the initial states( 1) (0.30,5.87,0.71,3.70,0.94,0.05,4.45,4.66) ,(0) (4.82,0.04,0.59, 4.91,1.17,1.75,5.63,0.82) .
T
T
xx− =
=
2 {6}S =
49
CASE III: Informed-Uninformed Local Rules
J. Lü, J. Liu, I. D. Couzin, S. A. Levin, Emerging collective behaviors of animal groups, WCICA, 2008, pp. 1060-1065.
50
Motivation:Motivation: Basic Local RulesBasic Local Rules
An illustration of the basic action mechanisms between individuals in MAS: A is the reciprocal action and B is the alignment.
Separation(Repulsion)
Cohesion (Attraction)
Alignment
AlignmentReciprocal action
Position information Direction information
51
Problem FormulationProblem Formulation
The sketch map of MAS and the information sources, where the golden ball is a MAS, the red and blue balls are two different information sources,respectively.
MAS
Information Source (point or direction)
Information Source (point or direction)
52
Updating Local RulesUpdating Local Rules
Consider the ability of grouping individuals to modifytheir motion on the basis of that of local neighbors.
Two fundamental actions:Reciprocal action:
Alignment:Unit direction vector
Unit vector from ci(t) to cj(t)
Reciprocal weight
Coupling coefficient
53
Updating Local RulesUpdating Local Rules
Each agent attempts to balance the influence ofits reciprocal action with that of its alignment. Thedesired travel direction of agent i at timeis given by
Reciprocal Action
AlignmentAlignmentRatio
54
A Simple Model With Information
Consider the information sources, the final traveldirection of agent i at time is described by
Desired Unit Vector
Preferred Unit Direction VectorWeight
Three balances in this model:
i) Repulsion Attraction
2) Reciprocal action Alignment
3) Desired direction Information direction
55
Numerical Simulations
The influence of the variation of the information sourceson the consensus of MAS.
MAS Initial information
Changed information
56
57
Outline
Introduction
Modeling, Analysis, Control of MAS
Some Advances in Consensus of MAS
Conclusions
58
Part IV: Conclusions
Estimate the convergence rate of MAS
Consensus of MAS with nonlinear local rules
Consensus of MAS with information
59
Some Future WorksSome Future Works
Consensus of MAS with Consensus of MAS with negative preferencenegative preference. This . This can be attribute to can be attribute to the convergence of generalized the convergence of generalized stochastic matricesstochastic matrices
Further investigation of Further investigation of nonnon--convex MAS modelsconvex MAS models, , trying to find a more universal method to tackletrying to find a more universal method to tacklethis kind of problemsthis kind of problems
