Collision avoidance and coalition formation of multiple unmanned … · 2013-02-11 · collision avoidance maneuver, in that case, is that which increases the line of sight rate between

Collision avoidance and coalition formation of multiple

unmanned aerial vehicles in high density traffic

environments

A Thesis

Submitted for the Degree of

Doctor of Philosophy

in the Faculty of Engineering

by

Joel George Manathara

Department of Aerospace Engineering

Indian Institute of Science

Bangalore – 560 012

May 2011

Abstract

There has been an increased interest in Unmanned Aerial Vehicles (UAVs) in the past few years

because of the wide variety of possible applications of UAVs in both military and civilian sectors.

Many missions involving UAVs are executed more efficiently when multiple UAVs are deployed,

and therefore an increased use of UAVs will cause certain regions of airspace to be cluttered with

UAVs executing various autonomous missions, leading to many operational issues. An important

consideration in such high density traffic airspace with UAVs in free flight is that the UAVs should

avoid colliding with other UAVs. Another problem is to form groups (coalitions) of UAVs whose

combined capabilities are required to accomplish certain missions. This thesis studies collision

avoidance and coalition formation of UAVs in free flight of high density traffic, and proposes

algorithms that can be easily implemented in realistic scenarios.

First, this thesis addresses the problem of collision avoidance among UAVs in an airspace

with high density traffic. Collision avoidance among aircraft has been extensively addressed in

the literature from the point of view of air traffic management. However, most of these works

consider collision avoidance among only a very few aircraft. This thesis considers the problem of

collision avoidance among multiple UAVs in situations where air traffic density is very high. The

main hypothesis proposed is: whenever a UAV encounters multiple conflicts, it identifies the ‘most

threatening’ conflict and does collision avoidance maneuver to avoid it. Every UAV doing so in

a multiple UAV free flight environment results in good performance in terms of lesser number of

near misses and smaller deviations from their nominal paths. The proposed algorithm considers

that conflict which has the least time-to-go to occur as the most threatening one. The appropriate

collision avoidance maneuver, in that case, is that which increases the line of sight rate between

the concerned UAVs. The efficacy of the proposed algorithm is tested using simulations involving

random flights with high density traffic. Also, simulations are done to show that the algorithm

performs well even in the presence of noise in position and velocity measurements of neighbor-

ing UAVs. This thesis also gives algorithms to handle collision avoidance among multiple UAVs

in three dimensions. Two 3-dimensional collision avoidance algorithms are developed, one of

which approximates the three dimensional engagements to planar conflicts and uses it for conflict

resolution, while the other handles three dimensional conflicts via deconfliction maneuvers that

are derived from the original three dimensional engagement geometries. Simulations are done to

evaluate the performance of these three dimensional collision avoidance algorithms. Further, the

proposed collision avoidance algorithms are implemented using realistic six degree of freedom

UAV models for which proportional-integral controllers are designed using successive loop clo-

sure method. Specific conflict cases as well as random flights in congested airspaces are studied

using these realistic UAV models and analyzed to show how the proposed algorithm handles those

i

situations. As the collision avoidance algorithms proposed in this thesis are reactive, they perform

well with heterogeneous – different speed – UAVs, and in cases where some of the UAVs in con-

flict are non-cooperative, as demonstrated though simulations. The studies in this thesis show that

the implementation of the proposed collision avoidance algorithms leads to a safer and efficient

operational airspace occupied by multiple UAVs.

Next, a search and prosecute mission involving a large number of UAVs and targets is consid-

ered. To prosecute a target, it is at times necessary to form a sub-group of UAVs, called a coalition,

whose combined resources suffice to prosecute the target. The task is to find and prosecute all the

targets in minimum time. It is proved in the thesis that the coalition formation problem addressed

is NP-hard. This thesis gives a sub-optimal coalition formation algorithm that has polynomial

time complexity. Since the algorithm has a polynomial time complexity, it scales well making it

suitable for cases where there are a large number of UAVs and targets. Simulations are carried

out to show that this coalition formation algorithm works well. The mission completion time us-

ing the proposed coalition formation algorithm is comparable to that of the solution of the global

combinatorial optimization problem obtained using Particle Swarm Optimization algorithm. Sim-

ulations show that the algorithm performs well in the presence of a large number of UAVs and

targets. The coalition formation algorithm is then extended to handle situations where the UAVs

have limited communication ranges. UAVs with limited communication ranges lead to a time

varying UAV network over which the UAVs have to communicate to form coalitions. Toward

this, a communication protocol is proposed which also takes into account the communication and

the computational delays associated with the UAV network. Also considered in this thesis, is the

prosecution of non-stationary and maneuvering targets. Simulation studies show that the proposed

algorithm is suitable for use in large UAV groups for real-time search and prosecute tasks.

Finally, this thesis considers some multiple UAV missions that require the application of colli-

sion avoidance and coalition formation techniques. Quite often, in multiple UAV missions, UAVs

are required to come together or rendezvous to exchange information and resources. The problem

of multiple UAV rendezvous is tackled by using a consensus among the UAVs to attain rendezvous

and the collision avoidance algorithm previously developed for safety. Simulations show how the

UAVs that deviated away from the preplanned trajectory, negotiating a conflict, still attain ren-

dezvous via velocity control demanded by the consensus algorithm. Also considered is a search

and prosecute mission where the UAVs also have to avoid collisions among each other. The colli-

sion avoidance algorithm developed is used in conjunction with the proposed coalition formation

algorithm to solve this problem. These studies demonstrate that the individual collision avoidance

and coalition formation algorithms, developed in this thesis, can be seamlessly integrated to work

together.

In summary, the main contributions of this thesis include (a) novel collision avoidance algo-

ii

rithms, which are conceptually simple and easy to implement, for resolving path conflicts – both

planar and three dimensional – in a high density traffic airspace with UAVs in free flight and (b)

efficient coalition formation algorithms for search and prosecute task with large number of UAVs

and targets where UAVs have limited communication ranges and targets are maneuvering. Sim-

ulations to evaluate the performance of algorithms based on these concepts to carry out realistic

tasks by UAV swarms are also given.

iii

iv

Acknowledgements

I am grateful to my thesis advisor Prof. Debasish Ghose for the great support and directions he

always provided, and the patience he had with me. I thankfully remember Late Dr. S. Pradeep

who was my thesis advisor during the initial two years of my doctoral studies. A part of the work

presented in this thesis was done over collaboration, and I thank Dr. P. B. Sujit who coordinated

the collaborations; and my collaborators Prof. F. L. Pereira, Prof. Randal Beard, Prof. J. B. Sousa,

and Dr. Jose Pinto. I thank Dr. Ambedkar Dukkipati for all his support and guidance. I am thankful

to everyone who, directly or indirectly, helped toward making the writing of this thesis possible.

v

vi

Contents

Abstract i

Acknowledgements v

List of Tables xi

List of Figures xiii

List of Algorithms xvii

Notations xix

1 Introduction 1

1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Collision Avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3 Coalition Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.4 Collision avoidance and coalition formation in multiple UAV missions . . . . . . . . . . 16

1.5 Contributions and organization of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . 17

1.6 A note to the reader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2 Collision avoidance among multiple UAVs 21

2.1 Preliminary Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.1.1 Some basic concepts and assumptions . . . . . . . . . . . . . . . . . . . . . . . 22

2.1.2 Overview of the proposed solution . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.2 Two dimensional collision avoidance algorithm . . . . . . . . . . . . . . . . . . . . . . . 24

2.2.1 Conflict detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.2.2 Collision avoidance maneuver . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.2.3 Effectiveness of handling pairwise conflicts . . . . . . . . . . . . . . . . . . . . 28

2.2.4 Dubins path to destination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

2.2.5 RCA-2D: two dimensional collision avoidance algorithm . . . . . . . . . . . . . 33

2.3 Three dimensional conflict avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2.3.1 Collision prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.3.2 Collision avoidance maneuver . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.3.3 Homing to destination in three dimension . . . . . . . . . . . . . . . . . . . . . 39

vii

2.4 Implementation of collision avoidance algorithms . . . . . . . . . . . . . . . . . . . . . . 39

2.4.1 Implementation of the RCA-2D algorithm . . . . . . . . . . . . . . . . . . . . . 39

2.4.2 Implementation of the RCA-3D-O algorithm . . . . . . . . . . . . . . . . . . . . 42

2.4.3 Implementation of the RCA-3D-I algorithm . . . . . . . . . . . . . . . . . . . . 43

2.4.4 Aircraft conflict resolution using Satisficing Game Theory (SGT) . . . . . . . . . 44

2.5 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

2.5.1 Simulation results for two dimensional case . . . . . . . . . . . . . . . . . . . . 48

2.5.2 Simulation results for 3D case . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

2.6 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

3 Collision avoidance with realistic UAV models 67

3.1 Six DoF equations of motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

3.2 Trimming aircraft for a level flight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3.3 Controller design for 2D collision avoidance . . . . . . . . . . . . . . . . . . . . . . . . 73

3.3.1 Altitude hold controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3.3.2 Velocity hold controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

3.3.3 Attitude hold controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

3.3.4 Testing the response of the UAV augmented with controller . . . . . . . . . . . . 75

3.4 Simulation results for planar collision avoidance . . . . . . . . . . . . . . . . . . . . . . 77

3.4.1 Tailored test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

3.4.2 Random flight test case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

3.5 Controller design for 3D collision avoidance . . . . . . . . . . . . . . . . . . . . . . . . 82

3.5.1 Six DoF model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3.5.2 Pitch rate guidance and control loops . . . . . . . . . . . . . . . . . . . . . . . . 83

3.5.3 Turn rate guidance and control loops . . . . . . . . . . . . . . . . . . . . . . . . 83

3.5.4 Response of UAV augmented with controller . . . . . . . . . . . . . . . . . . . . 83

3.6 Simulations results of 3D collision avoidance . . . . . . . . . . . . . . . . . . . . . . . . 86

3.6.1 Random flights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

3.6.2 Test case of modified random flights . . . . . . . . . . . . . . . . . . . . . . . . 88

3.6.3 Perpendicular flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

3.6.4 Non-cooperating UAVs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

3.6.5 Heterogeneous UAVs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

viii


4 Coalition formation with global communication 93

4.1 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

4.2 Coalition formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

4.2.1 Polynomial time coalition formation algorithm (PTCFA) . . . . . . . . . . . . . 103

4.2.2 Optimal coalition formation algorithm (OCFA) . . . . . . . . . . . . . . . . . . 109

4.2.3 Complexity analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

4.3 Simultaneous strike . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

4.4 Combinatorial optimization problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

4.4.1 Overview of particle swarm optimization . . . . . . . . . . . . . . . . . . . . . . 115

4.4.2 Solution to the optimal coalition formation problem using PSO . . . . . . . . . . 116

4.5 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

4.5.1 Solutions obtained using PSO and two-stage algorithms: an example . . . . . . . 122

4.5.2 Effect of increase in number of UAVs and targets . . . . . . . . . . . . . . . . . 129

4.5.3 Coalition formation for UAVs with heterogenous speed . . . . . . . . . . . . . . 136

4.5.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136


5 Coalition formation with limited communication 141

5.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

5.1.1 The mission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

5.1.2 Target and UAV kinematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

5.2 Communication protocol for coalition formation . . . . . . . . . . . . . . . . . . . . . . 145

5.2.1 Broadcast in a limited communication network . . . . . . . . . . . . . . . . . . . 145

5.2.2 Coalition formation over a dynamic network . . . . . . . . . . . . . . . . . . . . 147

5.2.3 Structure of the broadcast messages . . . . . . . . . . . . . . . . . . . . . . . . 148

5.3 Determining ETA to target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

5.3.1 Stationary targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

5.3.2 Constant velocity targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

5.3.3 Maneuvering targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

5.4 Prosecution of targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

5.4.1 Stationary target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

ix

5.4.2 Constant velocity targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

5.4.3 Maneuvering targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

5.5 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

5.5.1 Example scenario: stationary and constant velocity targets . . . . . . . . . . . . . 157

5.5.2 Example scenario: maneuvering targets . . . . . . . . . . . . . . . . . . . . . . 159

5.5.3 Effect of varying number of UAVs and targets . . . . . . . . . . . . . . . . . . . 162

5.5.4 Effect of varying communication range . . . . . . . . . . . . . . . . . . . . . . . 164

5.5.5 Effect of hop delay and max-hops . . . . . . . . . . . . . . . . . . . . . . . . . . 164


6 Collision avoidance and coalition formation in multiple UAV missions 169

6.1 Rendezvous via consensus without collision avoidance . . . . . . . . . . . . . . . . . . . 170

6.1.1 The consensus protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

6.1.2 Determining ETA to target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

6.1.3 Handling velocity bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

6.1.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

6.1.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

6.2 Rendezvous with collision avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

6.2.1 Incorporating collision avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . 177

6.2.2 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

6.3 Coalition formation in a search and prosecute mission with collision avoidance . . . . . . 181

6.3.1 Coalition formation in presence of collision avoidance . . . . . . . . . . . . . . . 182

6.3.2 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183


7 Conclusions 185

7.1 Collision Avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

7.2 Coalition Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

7.3 Collision avoidance and coalition formation in multiple UAV missions . . . . . . . . . . 189

7.4 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

Bibliography 191

Publications from this thesis 203

x

List of Tables

2.1 Comparison of performance of algorithms RCA-2D and SGT . . . . . . . . . . . . . 52

2.2 Comparison of computation times of RCA-2D and SGT per simulation . . . . . . . . 55

2.3 Comparison of performance of RCA-2D and SGT in the presence of noise in position

measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.4 Comparison of performance of RCA-2D and SGT in the presence of noise in heading

measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

2.5 Comparison of performance of algorithms RCA-2D and RCA-3D-O . . . . . . . . . 61

2.6 Comparison of computation times of algorithms RCA-2D and RCA-3D-O . . . . . . 61

2.7 Performance of the 3D algorithms for the test case of perturbed random flights . . . . 62

2.8 Performance of the 3D algorithms for the test case of modified random flights . . . . 63

3.1 The geometric and inertial parameters of the 6 DoF UAV model . . . . . . . . . . . . 71

3.2 Stability and control derivatives of the 6 DoF UAV model . . . . . . . . . . . . . . . 72

3.3 Initial positions and orientations of UAVs in Fig. 3.10 . . . . . . . . . . . . . . . . . 81

3.4 Random flight test case for 6 DoF UAV models with the inner and outer radii as 4 km

and 5 km . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

3.5 Random flight test case for 6 DoF UAV models with the inner and outer radii as 400 m

and 500 m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3.6 Random flight test case with 3D collision avoidance for 6 DoF UAV models . . . . . 88

3.7 Performance of the 3D algorithms in 6 DoF UAV models . . . . . . . . . . . . . . . 88

3.8 Trim conditions corresponding to various steady level cruise velocities . . . . . . . . 91

3.9 Random flight test case with 3D collision avoidance for heterogeneous 6 DoF UAV

models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

4.1 Details of the agents which responded to the coalition formation request in the given

example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

4.2 Details of the agents in the given example, after sorting . . . . . . . . . . . . . . . . 106

4.3 Comparison of the mission and simulation times obtained with the PSO solution and

the two-stage algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

6.1 Performance of the rendezvous algorithm in congested environments . . . . . . . . . 181

xi

xii

List of Figures

1.1 An example situation requiring collision avoidance . . . . . . . . . . . . . . . . . . 5

2.1 A 2D engagement between two UAVs . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.2 Variation of the LOS separations between two UAVs with time for various initial

engagement geometries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.3 Collision avoidance maneuver for a 2D engagement . . . . . . . . . . . . . . . . . . 30

2.4 The minimum LOS separations attained between two UAVs for various initial engage-

ment geometries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.5 Dubins paths to destination (the case of free terminal constraint) . . . . . . . . . . . 33

2.6 A sample situation requiring 3D collision avoidance among multiple UAVs . . . . . . 35

2.7 Engagement scenario and collision plane in a 3D conflict . . . . . . . . . . . . . . . 37

2.8 Collision avoidance in simple conflict situations . . . . . . . . . . . . . . . . . . . . 50

2.9 A schematic of the random flight test case . . . . . . . . . . . . . . . . . . . . . . . 51

2.10 Variation of near misses and efficiency with Rsen and Rdes . . . . . . . . . . . . . . . 53

2.11 A snapshot during the simulation of random flights . . . . . . . . . . . . . . . . . . 54

2.12 Scalability of number of near misses and efficiency with UAV density . . . . . . . . 57

2.13 Average decision time per simulation for different UAV densities . . . . . . . . . . . 58

2.14 Average decision time per UAV per second with increase in traffic density . . . . . . 59

2.15 A screenshot of UAVs making 3D collision avoidance maneuvers . . . . . . . . . . . 60

2.16 Collision avoidance in 3D using RCA-3D-O – the test case of perpendicular flow . . 62

2.17 The test case of modified random flights . . . . . . . . . . . . . . . . . . . . . . . . 63

3.1 Body axis of an aircraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3.2 Altitude hold controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

3.3 Velocity hold controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

3.4 Attitude hold controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

3.5 Response of 6 DoF UAV model to bank angle command . . . . . . . . . . . . . . . . 76

3.6 Turn rates for 6 DoF UAV model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

3.7 Collision avoidance of 3 realistic UAVs . . . . . . . . . . . . . . . . . . . . . . . . . 79

3.8 Control inputs and corresponding states for one of the UAVs in Fig. 3.7 . . . . . . . . 79


xiii


3.11 Pitch rate guidance and control loops . . . . . . . . . . . . . . . . . . . . . . . . . . 84

3.12 Turn rate guidance and control loops . . . . . . . . . . . . . . . . . . . . . . . . . . 85

3.13 UAV response to a doublet pitch rate demand . . . . . . . . . . . . . . . . . . . . . 86

3.14 UAV response to a doublet turn rate demand . . . . . . . . . . . . . . . . . . . . . . 87

3.15 Collision avoidance in three dimensional perpendicular flow . . . . . . . . . . . . . 89

3.16 The test case of array flow where none of the UAVs cooperate . . . . . . . . . . . . . 90

3.17 The test case of array flow where some of the UAVs cooperate . . . . . . . . . . . . 90

3.18 The test case of array flow where all the UAVs cooperate . . . . . . . . . . . . . . . 91

4.1 A situation where multiple agents detect multiple targets . . . . . . . . . . . . . . . 101

4.2 Sequence of events during coalition formation . . . . . . . . . . . . . . . . . . . . . 102

4.3 Dubins path and the roundabout path to a target . . . . . . . . . . . . . . . . . . . . 112

4.4 Calculation of length of roundabout path to target . . . . . . . . . . . . . . . . . . . 113

4.5 Initial positions of the UAVs and the targets for the PSO example . . . . . . . . . . . 120

4.6 The target assignment achieved using PSO in the given example . . . . . . . . . . . 121

4.7 Initial positions of the UAVs and the targets for the given example . . . . . . . . . . 123

4.8 The trajectory of U1 prosecuting T1 using PTCFA . . . . . . . . . . . . . . . . . . . 123

4.9 The trajectory of U1 prosecuting T1 using OCFA . . . . . . . . . . . . . . . . . . . . 124

4.10 Routes followed by the coalition U2,U3, and U4 formed by PTCFA to prosecute T4 . . 124

4.11 Routes followed by the coalition U3 and U4 formed by OCFA to prosecute T4 . . . . 125

4.12 UAVs U3 and U4 assigned to T3 by PTCFA . . . . . . . . . . . . . . . . . . . . . . . 125

4.13 UAV U2 assigned to T3 by OCFA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

4.14 Trajectories of UAVs U1 and U2 prosecuting target T2 using PTCFA . . . . . . . . . . 126

4.15 Trajectories of UAVs U1,U3, and U4 prosecuting target T2 using OCFA . . . . . . . . 127

4.16 Coalition formation and prosecution of all the targets in the given example using PSO 127

4.17 Comparison of the mean mission completion times (5 targets) . . . . . . . . . . . . . 130

4.18 Mean percentage mission not accomplished (5 targets) . . . . . . . . . . . . . . . . . 131

4.19 Comparison of mean mission completion times (10 targets) . . . . . . . . . . . . . . 131

4.20 Mean percentage mission not accomplished (10 targets) . . . . . . . . . . . . . . . . 132

4.21 Comparison of mean mission completion times (15 targets) . . . . . . . . . . . . . . 132

4.22 Mean percentage mission not accomplished (15 targets) . . . . . . . . . . . . . . . . 133

4.23 Average computational time taken to form coalitions using PTCFA and OCFA . . . . 133

xiv

4.24 Mean time taken to complete a simulation using PSO . . . . . . . . . . . . . . . . . 134

4.25 Initial UAV and target locations for the given example . . . . . . . . . . . . . . . . . 137

4.26 Trajectories of UAVs U4 and U5 to target T1 . . . . . . . . . . . . . . . . . . . . . . 137

4.27 Trajectories of UAVs U1,U2, and U3 to T3 . . . . . . . . . . . . . . . . . . . . . . . 138

4.28 Trajectories of UAVs U1,U3,U4, and U5 to T2 . . . . . . . . . . . . . . . . . . . . . 138

5.1 Search and prosecute mission using UAVs with limited sensor and communication

ranges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

5.2 Illustration of why Rcomm should be greater than 2Rsen . . . . . . . . . . . . . . . . . 143

5.3 Decision process and communication protocol for coalition formation . . . . . . . . 150

5.4 Dubins and roundabout paths to a moving target . . . . . . . . . . . . . . . . . . . . 152

5.5 Illustration of the iterative method to find ETA to a moving target . . . . . . . . . . . 152

5.6 Prosection strategy for maneuvering targets . . . . . . . . . . . . . . . . . . . . . . 156

5.7 Initial positions of the UAVs and the targets for the given example . . . . . . . . . . 158

5.8 Trajectories that the coalition members take to prosecute T1 . . . . . . . . . . . . . . 159

5.9 Trajectories that the coalition members take to prosecute the moving target T2 . . . . 160

5.10 Initial position of UAVs and targets for the given example . . . . . . . . . . . . . . . 160

5.11 Prosecution of T1 by U3 and U4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

5.12 A successful coalition for T2 is not formed due to lack of resources . . . . . . . . . . 162

5.13 Prosecution of T2 by U1 and U2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

5.14 Variation of mission time with increase in number of UAVs . . . . . . . . . . . . . . 164

5.15 Variation of mission time with increase in communication range of UAVs . . . . . . 165

5.16 Variation of mission time with increase in maximum hops allowed . . . . . . . . . . 165

5.17 Effect of Δhop and Hmax on mission performance . . . . . . . . . . . . . . . . . . . . 166

5.18 Variation of mission time with increase in communication delay . . . . . . . . . . . 167

6.1 Calculating the Dubins distance to a target . . . . . . . . . . . . . . . . . . . . . . . 172

6.2 Rendezvous of two UAVs using velocity control . . . . . . . . . . . . . . . . . . . . 173

6.3 Rendezvous of two UAVs via velocity control and a wandering maneuver . . . . . . 174

6.4 Rendezvous of 10 UAVs with velocity bounds . . . . . . . . . . . . . . . . . . . . . 175

6.5 Rendezvous of 3 UAVs onto a slowly maneuvering target . . . . . . . . . . . . . . . 175

6.6 Rendezvous of 20 UAVs using average consensus . . . . . . . . . . . . . . . . . . . 176

6.7 Rendezvous of 20 UAVs using weighted average consensus . . . . . . . . . . . . . . 177

6.8 Rendezvous of 20 UAVs using limited communication . . . . . . . . . . . . . . . . . 178

xv

6.9 Rendezvous of 20 UAVs using global communication . . . . . . . . . . . . . . . . . 178

6.10 Rendezvous of 5 UAVs with collision avoidance . . . . . . . . . . . . . . . . . . . . 179

6.11 The variation of ETAs of UAVs while attaining rendezvous in presence of collision

avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

6.12 Congested airspace in which a rendezvous need to be achieved . . . . . . . . . . . . 181

6.13 Rendezvous with and without collision avoidance in a dense environment . . . . . . 182

6.14 Trajectories of UAVs prosecuting a target with and without collision avoidance . . . . 183

6.15 Comparison of average mission times for a search and prosecute task with and without

collision avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

xvi

List of Algorithms

2.1 Deconfliction algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.2 Algorithm to implement Dubins path with free terminal constraints . . . . . . . . . . 32

2.3 Reactive Collision Avoidance 2D Algorithm . . . . . . . . . . . . . . . . . . . . . . 34

4.1 First stage of the PTCFA to form a minimum time coalition . . . . . . . . . . . . . . 104

4.2 Second stage of the PTCFA to prune members to obtain a reduced member coalition . 105

4.3 Optimal coalition formation algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 109

4.4 Global solution using PSO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

5.1 Algorithm to iteratively find the earliest interception point to a constant velocity target 153

xvii

xviii

Notations

Δhop Delay associated with each hop a message makes over network

λ Tuning parameter for algorithm RCA

ψi Heading of ith UAV

ψTj Heading of target Tj

θ LOS angle

ωmax Maximum turn rate of a UAV

ain Acceleration for deconfliction in the collision plane

aout Acceleration for deconfliction out of the collision plane

ai Acceleration of Ui

C (Ui,Tj) Set of UAVs in the coalition formed by Ui to prosecute Tj

D Destination

D(Ui,Tj) Set of distances to Tj of UAVs in C (Ui,Tj)

DTjUi

Roundabout path length from Ui to Tj

DTjUi

Dubins path length from Ui to Tj

ETAi Estimated time of arrival for Ui

Hmax Maximum number of hops a message is allowed to make

i, j Unit vectors

M Number of targets

Ni Set of neighbors of Ui

N Number of UAVs

n Vector perpendicular to collision plane

PC (Ui,Tj) UAVs that responded to coalition request of Ui for Tj

xix

pi Position of ith UAV, pi = (xi,yi,zi)

Q Dimension of each particle in PSO

RUi Resource capability vector of Ui, RUi = (RUi1 , . . . ,RUi

n )

RTi Resource capability vector of Ti, RTi = (RTi1 , . . . ,RTi

m)

R Radius of turn

Rcomm Communication range of UAV

Rdes Desired radius of turn

Rmin Minimum radius of turn

Rsen Sensor radius of UAV

r LOS separation

S Swarm size in PSO

τ Total time for mission completion

τTjc Time taken to prosecute Tj after coalition is formed

τTjUk

ETA of Uk to Tj

τTj(P,P′) Estimated time taken by target Tj to travel from point P to point P′

τUi(P,P′) Time taken by Ui to take a roundabout path from point P to point P′

τs Search time

Tj jth target

t time

tgo time to go

Ui ith UAV

V kl Velocity of lth particle at kth time step in PSO, V k

l = (vkl1,vk

l2, . . . ,vk

lQ)

VTj Velocity of target Tj

Vi Velocity of ith UAV

xx

Vmax Upper bound for UAV velocity

Vmin Lower bound for UAV velocity

vi Velocity vector of ith UAV, vi = (ui,vi,wi)

Xkl Position of lth particle at kth time step in PSO, Xk

l = (xkl1,xk

l2, . . . ,xk

lQ)

(xi,yi) Position of Ui

(xTj ,yTj) Position of Tj

Abbreviations

ATC Air Traffic Control

DoF Degree of Freedom

ETA Earliest/Estimated Time of Arrival

LOS Line of Sight

MAC Medium Access Control

OCFA Optimal Coalition Formation Algorithm

PI Proportional Integral

PN Proportional Navigation

PSO Particle Swarm Optimization

PTCFA Polynomial Time Coalition Formation Algorithm

RCA Reactive Collision Avoidance

SGT Satisficing Game Theory

TCP Transmission Control Protocol

TTL Time to Live

UAV Unmanned Aerial Vehicle

ZEM Zero Effort Miss

xxi

xxii

1 Introduction

This chapter introduces and motivates the problems that arise in missions involvingmultiple Unmanned Aerial Vehicles (UAVs). In particular, collision avoidance andcoalition formation of UAVs, which are the main topics of this thesis, are discussed.A review of existing literature is also given. The chapter concludes with an outlineof the thesis.

�n today’s world, when more and more tasks are automated and machines are

imparted intelligence to operate without human intervention, the Unmanned

Aerial Vehicles (UAVs) find a wide variety of applications like surveillance

and search. UAVs are pilotless aircraft which are partially or completely autonomous.

Research in UAVs is growing at a fast pace due to various capabilities that they offer

for military and civilian applications. Owing to its portability, low cost of acquisition,

operation, and maintenance, and absence to human risk, UAVs will continue to find

wider applications. The projected number of UAVs that will be in use in a decade from

now will be enormous certain tasks efficiently, and also to achieve robustness through

redundancy, it may at times be necessary to deploy a team of UAVs. Use of multiple

UAVs leads to several operational problems that need to be addressed. Among these,

this thesis addresses the problems of collision avoidance and coalition formation of

multiple UAVs in high density traffic environments, and their applications in multiple

UAV missions.

1.1 Motivation

Orville Wright, who flew the first recorded heavier than air powered flight in 1903,

remarked

“. . . it was nevertheless - the first time in the history of the world in which

a machine carrying a man had raised itself by its own power into the air in

full flight, had sailed forward without reduction of speed, and had finally

landed at a point as high as that from which it started.” (Wright, 1923)

The first flight, by Orville Wright, covered 120 feet (37 m) at a speed of only 6.8 miles

per hour (10.9 km/h) at about 10 feet (3.0 m) above the ground. Ever since that modest

1

first flight, the aircraft design industry has undergone an exponential rate of growth,

unparalleled by any other field of its time; except for the very recent advances in the

computer industry. Led by the slogan Faster, Farther, Higher (and Safer) (McMas-

ters & Cummings, 2002), over the years, the boundaries of speed, height, range and

endurance have been pushed farther and farther by aircraft designers. However, as in

the words of Orville Wright, aircraft always had had the tag of a “machine carrying a

man”.

Why UAVs?

Man has always been in the aircraft design loop. Design considerations included the

presence of pilot who will ultimately fly the aircraft. There is no reason to design

an aircraft of incredibly high endurance when the pilot suffers from fatigue in a much

shorter time. Similarly, there is no incentive to build a fighter aircraft that can take high

g-loads, which gives it a considerable advantage over the enemy aircraft, when the pilot

flying it ‘blacks out’ at much less tighter maneuvers. Motivated by these observations,

it is rational to conclude that the absence of a human in the aircraft cockpit throws

open the design space for new opportunities.

For example, the elimination of a pilot from a manned combat aircraft removes

many of the conventional design constraints like tolerance to g-loads, endurance limits,

safety factors, comfort, redundancy levels, and vulnerability. This at once throws open

the design parameter space; and dramatic improvements in performance measures like

increased speed, range, maneuverability, and payload can be achieved. Another exam-

ple is the coordinated turn maneuver. A coordinated turn requires the use of all the

aircraft controls – elevator, aileron, rudder, and throttle – to achieve a turn in which

the lateral velocities and forces are kept to zero. Attaintment of a coordinated turn

demands complicated control laws. However, a coordinated turn is insisted upon for

passenger comfort. In UAVs, as coordinated turn constraint is not critical, a turn can

be achieved by use of much simpler control laws.

Some missions, like oil pipeline monitoring, are monotonous jobs for humans and

they indeed do not require human presence. There are other missions, like nuclear

or chemical leakage inspection, that prohibits the presence of humans due to safety

reasons. Thus UAVs find great use in a wide variety of dull, dirty, and dangerous

missions.

2

Why autonomous UAVs?

Once the presence of a pilot is eliminated, the only ways to fly an aircraft is to do so

either remotely or autonomously. Flying an aircraft remotely has its own associated

problems like fatigue of the remote pilot, lack of adequate input and feedback for the

remote pilot, and the range limitation introduced by the necessity of communication

between a UAV and the remote pilot commands. Thus UAVs that can fly autonomously

is a more viable option. For example, a factor limiting longer endurance was the

presence of a human pilot. Introduction of autopilots for less critical long endurance

missions mitigate this problem.

As Rasmussen and Schumacher remarked in an editorial of a special issue on UAVs

(Rasmussen & Schumacher, 2007), “the lack of a human pilot,” on board or otherwise,

“enables a wealth of new operational paradigms.” However, higher levels of UAV au-

tonomy are required for the realization of these new operational capabilities. A clas-

sification of the levels of autonomy for UAV operations is given by Suresh and Ghose

(2010). Higher levels of autonomy in UAVs lead to greater flexibility and effectiveness

of UAV missions.

Why multiple autonomous UAVs?

Since UAVs are usually small owing to portability requirements, it is often necessary to

deploy a team of UAVs to accomplish certain missions. Also, there are some missions,

like search, that are more efficiently done by a group rather than a single UAV alone. In

addition, the use of multiple UAVs will add to missions, a robustness achieved through

redundancy. Thus, it is desirable for UAVs to work cooperatively in groups. The use

of sophisticated decentralized and cooperative control algorithms will enable groups

of UAVs to attain a higher level of ‘group autonomy’ (Suresh & Ghose, 2010), thus

attaining capabilities that significantly outperform that of individual UAVs.

Cooperation and coordination among multiple UAVs

The importance of the use of multiple UAVs and the necessity of undertaking a focused

research in this area was identified a decade ago (Banda, 2002). Although the use of

multiple UAVs for missions have several advantages, it will lead to many operational

issues. The UAVs will have to cooperate and coordinate with other UAVs, in the same

as well as other groups, to effectively carry out the respective missions. This is espe-

cially so in high density traffic situations. As Scerri et al. (2008) remark, “coordinating

hundreds or thousands of UAVs present a variety of new exciting challenges.”

3

The multiple UAV problems that requires cooperation and coordination include

cooperative path planning (Tsourdos et al., 2011; Shanmugavel et al., 2010; Kamal

et al., 2005), cooperative control (Tsourdos et al., 2007), and formation flying (Gu

et al., 2010; Kim et al., 2007).

One common subproblem in all multiple UAV missions is that the UAVs need to

avoid each other. The problem of collision avoidance among UAVs acquires great im-

portance in high density UAV traffic environments. This thesis addresses the problem

of collision avoidance among UAVs and gives algorithms which when implemented

by every UAV in a group results in safe missions.

Another problem of interest in multiple UAV missions, in general, in any multi-

agent systems, is that of coalition formation. Given a task, it is required to select a

subgroup, from a group of available UAVs, that has the required capabilities to com-

plete the task. This subgroup should also be the one that optimizes certain performance

measures. This thesis addresses a particular mission – search and prosecute mission –

that require coalition formation and provide algorithmic solutions.

Also addressed in the thesis are multiple UAV missions like rendezvous under

collision avoidance and coalition formation under collision avoidance requirements.

1.2 Collision Avoidance

In any multiple UAV mission, for its successful completion, the UAVs have to avoid

colliding with each other. The problem of multiple UAV collision avoidance can be

described as follows.

The problem

Consider a situation in which several UAVs are flying from different bases to their

respective destinations. These UAVs need to avoid mid-air collision with other UAVs

on their paths. For safety reasons, ‘miss distance’ or the minimum separation between

any two UAVs at any time of flight should be greater than a specified value. Any loca-

tion of two UAVs within the minimum required separation results in a near miss. The

objective is to find an algorithm that, when executed by every UAV, results in no colli-

sions and minimum number of ‘near misses’. Although it is desirable to have zero near

misses, it might be impossible to achieve this in high density air traffic scenarios like

the ones considered in this chapter. Moreover, in the case of UAVs, where there is no

4

U1

U2

U3

U4

U5

U6

safety zone

sensor range

Figure 1.1: An example situation requiring collision avoidance.

human risk, such a high level of safety requirement may not be necessary. So, the em-

phasis is on reducing the near misses while allowing some near misses, at the cost of

which a better efficiency can be obtained. Each UAV needs to achieve the above objec-

tive preferably in a decentralized manner. Situations in which communication among

UAVs may not be possible is considered. This prohibits cooperation via exchange of

information through communication between the UAVs. The collision avoidance ma-

neuvers need to be realistic and the maneuvers have to be efficient in the sense that the

deviation of a UAV from its nominal path due to collision avoidance maneuver should

be minimal to minimize late arrival at its destination. Each UAV needs to find a safe

path to its destination when limited information of positions and velocities, of only the

neighboring UAVs which are within its sensor range, is available.

Note that in the literature one may find the term collision avoidance being used for

the cases where UAVs have to avoid obstacles – both static and moving – which may,

more appropriately, be called obstacle avoidance. Although the problem of obstacle

avoidance can be handled by slight modifications of the collision avoidance algorithms

developed in this thesis, it is not of primary interest in this thesis.

Figure 1.1, which shows mid-air encounter of six UAVs during flight to their re-

spective destinations, gives an example scenario depicting the problem that we address

in this chapter. In the figure, the inner and the outer discs around UAV U1 show the

desired safety zone around the UAV and its sensor range. The same applies to all

other UAVs which have their own respective safety zones and sensor ranges. In this

particular situation, the UAVs U2, U4, and U5 are within the sensor range of UAV U1.

5

Also, UAV U1 may be on a collision course with U2 and U5. The problem of interest

is the development of a decentralized algorithm which, when employed by every UAV

with its limited sensor information, will result in safe paths for all the UAVs in a high

density traffic environment. Although it might appear that, since there is less freedom

for maneuver, in high density traffic situations it is appropriate and mandatory to use

a centralized coordination, Hoekstra et al. (2000), on the contrary, argue that “high

density situations are the ones that require the power of a distributed system.”

Collision avoidance in robotics

Collision avoidance has been an active area of research in the field of robotics. The

prominent ones are the dynamic window approach (Fox et al., 1997), the collision

cone method (Chakravarthy & Ghose, 1998), the velocity obstacle method (Fiorini

& Shiller, 1998; Shiller et al., 2001), and the inevitable collision states approach

(Fraichard & Asama, 2004; Gomez & Fraichard, 2009). These methods are devel-

oped with robotic application in mind. They perform better with more knowledge

about the obstacles (Gomez & Fraichard, 2009) (an information of the position, the

velocity, and the acceleration of an obstacle is considered as ‘more knowledge’ about

the obstacle than an information of its position and velocity alone). However, in cases

where a complete a priori knowledge of the path of an obstacle in not available, the

use of only current position and velocity values of obstacle can at some stage result

in very unrealistic changes in the magnitude and direction of demanded velocity. The

collision avoidance techniques employed widely in robotics, as mentioned above, are

typically computationally very intensive. This is acceptable for robots that can stop

to do the calculations, but not desirable for fixed wing UAVs that require a minimum

forward velocity to keep afloat in air. Since these algorithms are designed for robots,

it is assumed that the vehicles have braking forces and can stop if required and can

change their direction of motion instantaneously. Although these algorithms can be

generalized for multi-vehicle collision avoidance, they are originally designed for one

robot navigating through an environment with obstacles. Due to reasons described

above, the algorithms developed for robotic applications cannot be directly applied to

UAV problems.

Collision avoidance among aircraft/UAVs

There has been some research over the past decade on aircraft collision avoidance both

from the multiple UAV and the air traffic control points of view. All these collision

6

avoidance procedures are based on “See, Detect, and Avoid” principle (Lacher et al.,

2007). Extensive reviews of the collision avoidance algorithms are given from the air

traffic control point of view by Kuchar and Yang (2000); and by Albaker and Rahim

(2009b) from the UAV point of view. They classify the existing algorithms based on

(i) See: the type of sensing used, (ii) Detect: the conflict detection strategy used, and

(iii) Avoid: the selection of escape trajectories, maneuver realization, and the type of

multiple conflict management strategies (Kuchar & Yang, 2000; Albaker & Rahim,

2009b). They describe the possible ways in which collision avoidance algorithms can

differ, which is summarized as follows. The algorithms differ in the types of sensing

used – active or passive; and the sensing equipments used – radars, infrared sensors,

visual cameras, etc. The collision type considered can be planar or three dimensional.

Ways of collision detection include straight line projection of current velocities, con-

sideration of worst case maneuvers, probabilistic estimation of future locations, and

flight plan sharing between aircraft. The collision criterion may be time to collision,

point of closest approach, collision interval, etc. The type of maneuver used, once

a collision is predicted, can be predefined, protocol or rule based, reactive, potential

field based, optimized, or negotiation based. The execution of these maneuvers typ-

ically consist of the use of one or a combination of lateral-directional, longitudinal,

and/or velocity maneuvers. The collision avoidance algorithms also differ in the way

the multiple conflicts are handled. A brief overview of some of the collision avoidance

algorithms, which are relevant to the work in this thesis, is given below.

Collision avoidance in air traffic management

Most of the algorithms developed for air traffic management are those that guaran-

tee safe trajectories in a very low density traffic involving only two or three aircraft.

Tomlin et al. (1998) achieve conflict resolution by modeling the air traffic scenario as

a multi-agent hybrid system. The method developed in (Tomlin et al., 1998), which

is called the ‘roundabout’ method, ensures safe path for cases of two or three air-

craft only. A generalized roundabout method was developed by Frazzoli et al. (2005)

in which a decentralized hybrid control policy is used and is shown that the method

works for higher number of aircraft. In both the approaches, the efficiencies achieved

are poor due to the roundabout paths taken.

Minimum length conflict free paths are obtained by Bicchi and Pallottino (2000)

using optimization methods. As this algorithm is not decentralized, it is implementable

for only a few aircraft. Bilimoria (2000) provides a geometric optimization approach

for conflict resolution. Hwang and Tomlin (2002) propose a protocol-based conflict

7

resolution in air traffic scenarios with finite sensor radius limits. This algorithm in-

volves instantaneous velocity and heading changes, and thus might result in unrealis-

tic maneuvers. Mao et al. (2000) give a collision avoidance algorithm for intersecting

flow of aircraft. However, their algorithm also requires instantaneous changes in po-

sition and heading of aircraft which is unrealistic. Also, it is not applicable in highly

dynamic environments considered in this thesis.

Negotiation based collision avoidance

There are certain algorithms that consider aircraft as agents and deconfliction, in case

of a detected conflict, is achieved by peer to peer negotiation (Sislak et al., 2006,

2007; Albaker & Rahim, 2009a). The aircraft share their flight plans, and modify

them based on negotiation to find safe trajectories. Such an approach can be used for

air traffic management application as well as for UAV collision avoidance. In (Sislak

et al., 2006), the conflict resolution is rule-based and depends on the angle of predicted

collision between the aircraft. This algorithm is more suited for air traffic management

application rather than UAV collision avoidance. Sislak et al. (2007), for each conflict,

evaluate all possible predefined set of maneuvers for the best solution. This approach

can turn out to be computationally intensive. They consider a test case equivalent to the

random flight test case considered in this thesis to test the collision avoidance algorithm

for a high density traffic. Albaker and Rahim (2009a) use a straight projection for

conflict detection and predefined heading and speed changes for deconfliction using

a cooperative peer to peer negotiation. Negotiation requires communication between

aircraft. Therefore, these algorithms cannot be used in adversary environments where

communication may not be possible. Samek et al. (2007) use a multi-party negotiator

instead of a peer to peer negotiation. Simulations they present involve high density

traffic. In this approach, several UAVs having mutual conflicts are grouped together.

This gives the benefits of a centralized solution. The communication and computation

overhead associated with this collision avoidance algorithm is high.

Collision avoidance using artificial potential based methods

Another approach to collision avoidance is artificial potential based methods where

aircraft or UAVs are treated like ‘charged particles’ of same charge that repel each

other; the destination of an aircraft is modeled as a charge of the opposite sign so as to

attract or navigate the aircraft toward the destination. The artificial potential methods

are susceptible to local minima and require breaking forces, and therefore is not widely

8

used in UAV collision avoidance. However, several variants of potential based meth-

ods have been used for aircraft and UAV collision avoidance. Eby (1994), Eby and

Kelly (1999) give a potential field like method where once a violated separation is de-

tected, the velocity vector is changed. This change (acceleration) is proportional to the

predicted miss distance and inversely proportional to the time-to-go. These papers give

simulation results for low density flight scenarios. They study the performance of their

algorithm in the presence of degraded communication and maneuverability. Roussos

et al. (2010) use artificial potential field generated by ‘dipolar navigation functions’ to

effect a collision avoidance. They use a nonholonomic vehicle model with a constraint

on the motion along lateral axis, avoid high yaw rates, and do collision avoidance

simulations for six aircraft. Shim et al. (2003) use a potential function method along

with nonlinear model predictive control to address the problem of collision avoidance

in three dimensions. They demonstrated, in simulation, collision avoidance between

five helicopters which are modeled using six degrees of freedom and realistic con-

trol inputs. Another limitation with the potential field methods is that the computed

solutions are not guaranteed to be feasible once the aircraft dynamic limitations are

imposed. Rahmani et al. (2008), Panyakeow and Mesbahi (2010) use a navigation

function along with a swirling function to take care of constant velocity and turn rate

limits in UAVs, and the mission requirements are incorporated in the potential func-

tions. The potential methods, typically, use only the position information. Ignoring

the velocity information can lead to highly conservative solutions that are inefficient.

Potential function based methods are also computationally intensive.

Collision avoidance based on collision cone approach

Chakravarthy and Ghose (1998) developed the concept of ‘collision cone’ that can

be used to determine whether two objects, of irregular shapes and arbitrary sizes, are

on a collision course. The collision cone approach has been the basis for many colli-

sion/obstacle avoidance algorithms. Since an aircraft with a safety ball around it can be

considered as a moving obstacle of the size of the safety ball, modifications of the col-

lision cone approach can be used for aircraft deconfliction. Lalish et al. (2008), Lalish

(2009) use the collision cone approach in conjunction with a control function which

ensures collision free trajectories in a multi-vehicle scenario with no sensor range lim-

itations, provided the vehicles start in a conflict free situation. This distributed reactive

collision avoidance algorithm applies to both planar and three dimensional engagement

of aerial vehicles (Lalish, 2009). If the vehicles are in a deconflicted state initially, the

algorithm ensures the deconfliction at all later times. The algorithm is not decentral-

9

ized as a UAV implementing this algorithm requires information of all other UAVs.

This may, in general, be not possible and might lead to intensive computational re-

quirements in high density traffic environments. Also, the ‘guaranteed’ deconfliction

can at times be sacrificed for efficiency; an effort to strictly stick to assured separation

at times can lead to very inefficient trajectories. Such a situation can occur more fre-

quently in highly cluttered environments which they do not consider. Another method

based on collision cone approach is due to Smith and Harmon (2010). They make use

of the orientation rate to get a converging (or diverging) cone. A True Proportional

Navigation guidance law is used to drive the velocity vector out of the cone. The al-

gorithm was tested for different engagement geometries of two or three UAVs only.

Nonetheless, they do software-in-loop, hardware-in-loop, as well as real flight tests of

their collision avoidance algorithm.

Collision avoidance based on pairwise deconfliction and geometry

This thesis advocates the handling of multiple conflicts by ‘appropriately’ handling

pairwise conflicts. Therefore, we review some of the one-one deconfliction algorithms

in the literature. Among these are certain collision avoidance optimal algorithms for

pairwise deconfliction. Han and Bang (2004) give a Proportional Navigation (PN)

based algorithm for optimal collision avoidance among two aircraft. Merz (1991)

calculates the optimal maneuver to maximize the miss distance between two aircraft.

Using tools of optimal control, he shows that an acceleration along the predicted near

miss direction is the one that maximizes near miss. Such an acceleration is achieved

through thrust, break, climb, dive, and turns. Park et al. (2008) also use a change in

velocity vector to effect an increase in near miss in the direction of predicted near miss

for a pair of UAVs. Carbone et al. (2006) gives three dimensional geometric algo-

rithm for optimal deconfliction among two aircraft. They provide analytic solutions

for lateral-directional, longitudinal, and speed only control. The most effective control

for collision avoidance is found to be the lateral-directional maneuvers. Their studies

show that the speed only control is not very effective. This justifies the use of constant

velocity maneuvers used in this thesis to effect deconflictions. Most of the above colli-

sion avoidance algorithms use a geometric approach. These algorithms are optimal in

the sense that the velocity change required is minimal or the deviation from the nom-

inal path is minimal. However, the optimality of such algorithms creates a barrier for

them from being used for multiple UAV collision avoidance. Since these algorithms

are optimal only for a pair, they fail in the presence of multiple conflicts.

10

Dowek et al. (2005) develop a pairwise distributed resolution strategy where the

resolution maneuver is either change in vertical speed, heading, or ground speed. They

perform simulations for very high density traffic and report good results. However, no

information on the efficiency of this algorithm is available.

Shin et al. (2008), Tsourdos et al. (2011) give a multiple UAV collision avoidance

algorithm where geometric methods are used to detect pairwise conflict, and then use

a direction or velocity control to effect a deconfliction. They also study the stability of

these conflict resolution strategies.

Game theory based collision avoidance

Hill et al. (2005) and Archibald et al. (2008) use satisficing game theory to address the

problem of collision avoidance in multiple UAVs. These papers present simulations

involving test cases with high density air traffic scenarios to test the proposed algo-

rithm. This algorithm, apart from being computationally intensive, requires constant

communication between neighboring UAVs which may not always be possible. In the

sequel, this algorithm is referred to as Satisficing Game Theory (SGT) based algo-

rithm for collision avoidance and is used for comparison studies with the algorithms

developed in this thesis.

1.3 Coalition Formation

In recent times, there has been some instances of the use of UAVs for search and

prosecute missions. In a typical search and prosecute mission, multiple UAVs are

deployed that cooperate with each other as a team to detect and subsequently prosecute

the detected static and moving targets in a region of interest. Deploying multiple UAVs

for the search and prosecute mission provides robustness to the mission and reduces

the mission completion time.

The problem

Often, UAVs may have to use various types and quantities of resources to completely

prosecute a target. A single UAV may not have sufficient resources to prosecute the tar-

get, and hence a sub-team of UAVs, whose total resources are sufficient to completely

prosecute the target, needs to be formed. This sub-team of UAVs is called a coalition;

and each UAV in the coalition, a coalition member. The UAV that detected the target

11

initiates the coalition formation process, forms the coalition, and is called the coalition

leader. While the primary objective of the mission is to search and prosecute all the

targets, the other objectives are: (i) prosecute the target in minimum time, (ii) mini-

mize the coalition size, and (iii) simultaneously prosecute the target (if possible). The

objective (i) enables the UAVs to quickly accomplish the mission. The requirement

of objective (ii) can be justified as follows. The UAVs have limited sensor range and

therefore need to distribute their search effort efficiently to quickly detect the target. A

small coalition will allow the rest of the UAVs to search for targets resulting in quicker

target detection, while a larger coalition will leave only a few UAVs for distributed

search task, thus reducing the total search effort and increasing the mission time. The

objective (ii) implicitly assists in completing the mission quickly. The objective (iii) is

required to induce maximum damage to the target.

Additionally, the resources of the UAVs, such as fuel and weapons, deplete with

use and the UAVs have kinematic constraints like minimum radius of turn and veloc-

ity bounds. Since the UAVs are in motion, for the coalitions formed to be efficient,

the coalition formation algorithms must be quick with low computational overhead.

Therefore, there is a need to develop coalition formation algorithms that have low

computational complexity, satisfy the objectives (i)–(iii), and take the depletion of re-

sources and kinematic constraints into account.

Gerkey and Mataric (2004) present a taxonomy for the multi-robot task allocation

problem. The problem of coalition formation for a search and prosecute task, accord-

ing to their taxonomy, falls in the Multi Task - Multi Robot (MT-MR) instantaneous

assignment problem category because the UAVs can perform multiple tasks, namely

search and target prosecution tasks, simultaneously, and the targets may require mul-

tiple UAVs to prosecute. This thesis, in particular, addresses the coalition formation

for a search and prosecute mission. However, the abstract formalism adopted in this

thesis toward formulation and solution of the coalition formation problem makes it

applicable to a general task allocation problem.

Although we treat the UAV resources as abstract entities in this thesis, in actuality,

the UAV resource capabilities can include, but are not limited to, visual sensing power

(cameras and radars), weapon power, speed, and maneuverability. Thus, an example

of a multiple UAV mission that requires coalition is tracking and prosecuting a moving

target. Such a task requires UAVs with high visibility sensors that can track the target

and UAVs with adequate weaponry that can destroy the target.

12

Coalition formation in multi-agent systems

In general, a coalition is a group of team members that have agreed to cooperate with

each other to execute a single task (Shehory & Kraus, 1998). The coalitions formed

are temporary by nature; once the task is accomplished, the coalition members can

perform other tasks. Forming a coalition to achieve tasks is an active field of research

in the multi-agent community (Sandholm et al., 1999; Shehory et al., 1998) and the

multi-robot community (Vig & Adams, 2006a, 2005; Parker & Tang, 2006). Deter-

mining the optimal coalition from a group of agents is a computationally intensive

task and is NP-hard (Sandholm et al., 1999). Therefore, the effort has been to find

algorithms that provide approximate, near-optimal, and anytime solutions (Shehory &

Kraus, 1998; Sandholm et al., 1999; Vig & Adams, 2006b; Rahwan et al., 2009).

In multi-agent system, the agents are usually software, their resource being pieces

of codes or data, and task being computation or data mining. Campbell et al. (2002)

considers a task allocation problem in which a group of agents is presented with a se-

quence of tasks, and each agent chooses whether to join a coalition to solve a particular

task. This is typical of most of the coalition formation algorithms. Coalition formation

at large scales have been looked at by Scerri et al. (2008) and Tosic and Agha (2005).

Scerri et al. (2008) considers the problem of coordination of two hundred wide area

search munitions or UAVs to find and destroy ground based targets. However, their

work is more from the point of view of multi-agent systems as they do not consider

the dynamics or kinematics of the UAVs.

Coalition formation in robotics

The coalition formation algorithms developed in the multi-agent community cannot

be directly applied to multiple robot systems (Vig & Adams, 2006a) nor to the multi-

ple UAV systems. This is because, there is a difference in the kind of resources used

by software agents and robots. For software agents, the resource is typically a piece

of code or some data which is transferable between the agents. Whereas, for robots

or UAVs, the resources are sensors, actuators, et cetera which cannot be transferred

between them. The cooperation among UAVs or robots is achieved through informa-

tion exchange and their physical effect on the environment. In multi-agent systems, the

coalition structure depends on the member alone and not on the allocated task, whereas

in multi-robots systems, the purpose of coalition formation itself is task allocation (Vig

& Adams, 2006a).

Vig and Adams (2006a, 2005) develop a coalition formation scheme, where the

13

tasks act as agents and perform the function of an auctioneer for gathering bids and

determining the coalition. This process of forming coalitions is different from the

typical approach where a robot is an auctioneer. In (Vig & Adams, 2006a), the authors

use a negotiation process to form the coalition. It is well known that the negotiation

process requires significant amount of communication and also takes time to form

the coalition. The authors of (Vig & Adams, 2005) pose the problem as a matching

problem that is also computationally intensive. The same authors address the issue of

balance in the coalition between the coalitional size and the resource contribution in

(Vig & Adams, 2006a). The algorithms developed in (Vig & Adams, 2006a), (Vig &

Adams, 2005), or (Vig & Adams, 2006a) require significant amount of computation

time and communication which may not be possible in UAV networks as UAVs move

fast and cannot stop in mid air. Hence, these algorithms cannot be directly applied

to the UAV scenario. Parker and Tang (2006) present a coalition formation scheme

where a coalition leader robot broadcasts the existence of a task and other robots reply

by providing their availability. The leader robot evaluates all possible coalitions and

sends an accept decision to the robots that it considers suitable. The task is executed by

sharing the sensor information. However, they do not take coalition size or minimum

arrival time to the target into account.

Coalition formation in UAV groups

The problem of allocating tasks to UAVs is similar to the multi-robot task allocation

problem. However, there are two differences: first, UAVs travel with greater veloci-

ties than ground robots and cannot stop in midair; and secondly, the resources present

in the robots do not deplete with use or can be easily replenished, whereas UAV re-

sources, like fuel and weapons, deplete with use. Thus, the multi-robot task allocation

algorithms cannot be directly used for multiple UAV missions requiring coalition for-

mation.

The problem of forming a coalition has not been adequately addressed in the UAV

community. Nonetheless there are important contributions some of which we describe

below. Kingston and Schumacher (2005) assign multiple UAVs to track and prosecute

a target using an mixed integer linear programming formulation. The goal is to mini-

mize the length of task tours of the UAVs with known target positions. The resources

of the tracking agents have the same capability and the problem of depleting resources

is not addressed. Many researchers have developed task allocation algorithms for ef-

ficiently allocating UAVs to different tasks. Most of the solutions to task allocation

problems assume the UAVs (a) are homogenous, (b) can carry resources that do not

14

deplete with use, (c) can individually prosecute the target, and (d) can prosecute the

target with any resource (Nygard et al., 2001; Schumacher & Chandler, 2004; Darrah

et al., 2006; Alighanbari & How, 2006; Sujit et al., 2005). Therefore, we cannot use

these algorithms directly to the search and prosecute problem as posed above.

Some works in the literature consider optimal weapon-target or vehicle-target as-

signment problem that was shown to be NP-complete by Murphey (1999). Arslan

et al. (2007) address a generalized version of this problem using game theory, model-

ing it as a potential game, and solving it via the design of appropriate utility functions.

Shima et al. (2009) use mixed integer linear programming, tree search, and genetic

algorithms for the optimal vehicle-target assignment problem in UAV groups. They

consider depletion of resources, like fuel, but do not present a case where resource

limits are reached. The inherent assumption in the above papers is that a search of

the area of interest has already been carried out and the target locations and types are

identified. However, this assumptions limits the use of such algorithms to stationary

targets and therefore cannot be used in the presence of multiple moving/maneuvering

targets as considered in the search and prosecute task that this thesis addresses.

Coalition formation under limited communication

When UAVs with limited communication ranges are considered, the problem of coali-

tion formation assumes a new dimension. The coalition formation process, now, has

to be carried out over a time varying network formed by the UAVs moving around in

search of targets. An additional complexity arises due to the presence of moving and

randomly maneuvering targets. In the approaches described above where coalitions

are formed over groups of communicating robots, the robots do not face a situation

where they can break the communication network during the coalition formation pro-

cess since the robots can stop and form a static network until the coalition formation

process is completed. However, this is not possible in multiple fixed-wing UAV groups

that require a minimum velocity to be airborne.

Toward solving the problem of forming coalitions over a time varying network,

we, in this thesis, use TCP/IP (Transmission Control Protocol/Internet Protocol) like

protocols. Vanzin and Barber (2006) determine a mechanism of finding potential mem-

bers for coalition based on a TCP protocol. However, the technique does not address

the coalition formation issues. Although their algorithm allows for the agents to join

and leave the network at will, they do not consider networks where topology changes

due to the agents moving around. This thesis develops new techniques to determine

coalition in a dynamic network.

15

1.4 Collision avoidance and coalition formation in multiple UAV missions

Collision avoidance and coalition formation find immense applications in a wide vari-

ety of multiple UAV missions. In multiple UAV missions, it is common that the UAVs

are required to rendezvous at a point or a region to exchange confidential informa-

tion and/or share resources. This thesis considers multiple UAV rendezvous problem

where a rendezvous to a predetermined point is desired while the UAVs involved in the

operation have to do evasion maneuvers to avoid collision en route to the rendezvous

point.

Previous works address the multiple agent rendezvous problem in which multiple

agents (not necessarily UAVs) arrive simultaneously to a common location by cooper-

ating with each other (Lin et al., 2003; Tiwari et al., 2004; Lin et al., 2005; Notarste-

fano & Bullo, 2006; Das & Ghose, 2009). In these papers, rendezvous is achieved

via consensus between the agents on their positions and therefore rendezvous does not

occur at a predetermined point but is a function of the initial positions of the agents.

Sinha and Ghose (2006) use a linear cyclic pursuit of agents to attain rendezvous. They

show that, with an appropriate choice of controller gains, a rendezvous to a desired

point can be achieved. All the above papers do not consider the kinematic constraints

of UAVs like turn rate limitations and velocity bounds. Linear cyclic pursuit can be

extended to a nonlinear cyclic pursuit by adding nonholonomic constraint of minimum

radius of turn. In that case, Sinha and Ghose (2007) show that the cyclic pursuit can-

not achieve rendezvous at a point but rather will result in a ‘rendezvous about a point’.

Furukawa et al. (2005) require multiple UAVs to simultaneously arrive at a target. This

is achieved using time-optimal control of orientation and velocity. Although they ac-

count for turn rate limitations and velocity bounds present in a realistic UAV, they do

not consider any obstacle/collision avoidance. McLain and Beard (2005) solve the

optimization problem of minimizing time to rendezvous point, of a group of realis-

tic UAVs, as well as their times of exposure to known threats. However, they do not

consider collision avoidance between UAVs in free flight.

Tsourdos et al. (2005), Shanmugavel et al. (2006), Tsourdos et al. (2011) consider

a problem where multiple UAVs have to arrive at their respective destinations, simul-

taneously; the UAVs have to do so while avoiding collisions among themselves and

with obstacles. The authors of the above cited papers give solutions for this problem

based on Dubins, clothoid, and Pythagorean hodograph paths. Although these papers

consider rendezvous only in time and not in space, unlike the problem that we consider

16

in this thesis, similar techniques can be used to address the problem where UAVs have

to rendezvous not only in time but also in space.

The multiple UAV rendezvous algorithm developed in this thesis can be easily

modified for standoff tracking of stationary and moving targets. Standoff tracking finds

applications like surveillance, reconnaissance, and convey protection, among others.

The approach in the literature toward this problem is to use Lyapunov guidance vec-

tor fields (Frew et al., 2008; Summers et al., 2009). The consensus based algorithm

provided in this thesis is conceptually simpler than the methods given by above papers.

Another multiple UAV mission this thesis addresses is that of coalition formation

for a search and prosecute task in the presence of collision avoidance. This problem is

novel and has not been addressed before in the literature.

1.5 Contributions and organization of the thesis

We briefly give the outline of the thesis highlighting the major contributions.

In Chapter 2, we address the problem of collision avoidance among UAVs in

high density traffic situations, and develop simple and efficient algorithms for conflict

resolution. First, an algorithm is developed to handle planar conflicts. This thesis han-

dles multiple UAV collisions by appropriately handling pairwise conflicts, and uses

principles from missile guidance to effect conflict resolution. It is shown, through

simulations, that the proposed conflict resolution strategy handles pairwise conflicts

correctly. Simulations involving high density traffic are presented to demonstrate the

efficacy of this algorithm. Next, we develop algorithms to handle conflicts that are

inherently three dimensional. Two algorithms are developed to address three dimen-

sional conflicts, one of which is the extension of the proposed planar algorithm. Again,

through simulations, we show that these algorithms perform well in high density traffic

environments that are of interest to this thesis.

In Chapter 3, we demonstrate the ease of implementation of the conflict resolu-

tion algorithms developed in Chapter 2. Six degrees of freedom UAV models are used

as the platform to implement the collision avoidance algorithms. For planar collision

avoidance, separate altitude, attitude, and velocity hold controllers are designed, using

successive loop closure technique, to realize the guidance commands issued by the col-

lision avoidance algorithm. Flight path angle, course, and velocity hold controllers are

used to implement the three dimensional collision avoidance algorithm. These realis-

tic UAV models, augmented with controllers, using the collision avoidance algorithms

17

developed in this thesis are shown to have good collision avoidance performance in

simulations involving very high density traffic.

In Chapter 4 we consider the problem of coalition formation among multiple

UAVs for a search and prosecute mission. First, we prove that the coalition forma-

tion problem that we are tackling is indeed NP-hard. Then, a two stage polynomial

time algorithm is developed to give sub-optimal, but computationally efficient and

simple, solution to the coalition formation problem involving large number of targets

and UAVs. We solve the global optimization problem of target prosecution, when

the target locations are know a priori, using Particle Swarm Optimization algorithm.

Knowing the locations of targets a priori makes the problem that of optimal sequenc-

ing, with additional complexities introduced by resource requirements and resource

depletion. Comparison of the results shows the near optimality of the polynomial time

coalition formation algorithm that we propose.

In Chapter 5, we consider the coalition formation problem for search and prose-

cute mission when the UAVs have limited communication ranges. This situation will

occur in missions with large number of small UAVs. The UAVs being small will have

small communication ranges causing a dynamic communication network to be formed

by the UAVs exploring the environment in search of targets. In this case, we give a

communication protocol that enables the UAVs to form coalitions over time varying

networks. We study the dependence of the performance of this protocol on the mis-

sion and the network parameters, and show that our algorithm can be used in UAV

dense environments. In this chapter, we also consider the case where the targets are

moving/maneuvering.

In Chapter 6, we consider certain multiple UAV missions that require collision

avoidance and/or coalition formation among UAVs for the successful completion of

the mission. First, a multiple UAV rendezvous problem is considered under collision

avoidance. A consensus based algorithm that we develop makes rendezvous possible,

while the collision avoidance algorithm of Chapter 2 takes care of the conflicts en

route rendezvous. Next, we consider coalition formation under collision avoidance.

Simulations show that the collision avoidance and the coalition formation algorithms

that we developed can be seamlessly integrated into UAVs, for multiple UAV missions,

depending on the mission requirements without much interference between them.

Chapter 7 concludes the thesis with a summary, discussion of main results, and

directions for future research.

18

1.6 A note to the reader

All symbols and abbreviations used in the thesis are defined at the place of their first

use. For the convenience of the reader, the commonly used notations in the thesis

are given in the preamble. Some of the notations that we use are standard in the

literature. For example, the symbols used to denote the stability and control derivatives

in Chapter 3 are standard in the flight dynamics community, for instance (Stevens &

Lewis, 2003). Some notations are specific to a particular section of the thesis. At

certain places in the thesis, we use the same notation for different concepts if this does

not cause ambiguity, and in such cases, the correspondence is made clear from the

context. Since a multiple UAV system can be looked upon as a multi-agent system, at

some places in the thesis, the words agent and UAV are used interchangeably.

19

20

2 Collision avoidance among multiple UAVs

This chapter develops reactive algorithms for collision avoidance among multipleUAVs for both two and three dimensional conflicts. The main idea proposed isthat the collision avoidance among multiple UAVs can be achieved by effectivelyhandling the pairwise conflicts. The performance of the algorithms developed areanalyzed through extensive simulations.

�ultiple UAV missions lead to many emergent subproblems that need to be

addressed effectively. For any mission involving multiple UAVs, a com-

mon subproblem is that the UAVs may collide with each other. UAVs oc-

cupying the same airspace need to avoid each other to ensure a safe completion of

respective missions. The problem assumes great importance in a high density mul-

tiple UAV traffic scenario. In this chapter, we develop a simple and novel collision

avoidance algorithm to resolve conflict among UAVs that are on collision course in

an airspace occupied by a large number of UAVs. These algorithms are conceptu-

ally simple, computationally efficient and easily implementable. We show the efficacy

of the developed collision avoidance algorithms through simulations using kinematic

UAV models and high density air traffic scenarios where most of the possible conflict

geometries are expected to occur.

The organization of the chapter is as follows. Section 2.1 discusses some basic

concepts, issues, and assumptions, and gives an overview of our solution to the multi-

ple UAV collision avoidance problem. In Section 2.2 we develop a collision avoidance

algorithm to resolve planar conflicts. The algorithm is extended to handle 3D conflicts

in Section 2.3 and then a genuinely 3D collision avoidance algorithm is proposed. In

Section 2.4 we give the implementation of these algorithms. Section 2.5 presents the

simulation results. In Section 2.6 we conclude the chapter with a summary of the main

contributions.

2.1 Preliminary Discussion

First, we discuss some basic concept, issues, and assumptions, and provide an outline

of the proposed solution.

21

2.1.1 Some basic concepts and assumptions

We make the following assumptions regarding the kinematics of a UAV and its sensor

capabilities. It is assumed that a UAV has limited sensor range and knows exactly the

positions and velocities of all UAVs within its sensor range. However, through simu-

lations, we show that the collision avoidance algorithm that we develop in this chapter

also perform well in the presence of noise in position and velocity measurements. Al-

though it is assumed that the UAVs have sensors, in situations where it is desirable

to turn off the sensors to avoid getting detected, the UAVs can get information about

their neighbors through inter-UAV communication. This is possible in a cooperative

environment. Broadcast mechanisms like automatic dependent surveillance-broadcast

(ADS-B) (RTCA, 2002) can also be used in such situations, if the UAVs are ADS-B

enabled. We consider UAVs with a kinematic constraint of minimum radius of turn

which implies that a UAV cannot instantaneously change its heading.

Initially, we assume a kinematic model for UAVs as given below. However, in the

next chapter, we consider a realistic UAV model with six degree of freedom dynamics.

The kinematics of the UAV Ui with position (xi,yi), velocity Vi, and heading ψi is

xi = Vi cosψi

yi = Vi sinψi

ψi = ai/Vi (2.1)

where, ai is the commanded lateral acceleration issued by the collision avoidance guid-

ance algorithm. If Rmini is the minimum radius of turn, then |ai| ≤ V 2i /Rmini . The set

of equations in Eq. (2.1) give the two dimensional kinematics. In three dimensional

case too, we assume a similar kinematics where a lateral acceleration perpendicular to

the velocity vector provides the required turn rate. The details of two and three dimen-

sional kinematics used for simulations in this chapter are given in Section 2.4, which

explains the implementation details of the collision avoidance algorithms proposed in

this thesis.

2.1.2 Overview of the proposed solution

A broad overview of the solution that we propose in this chapter for the multiple UAV

collision avoidance, while the UAVs fly from their bases to respective destinations,

is as follows: When a UAV envisages a near miss with another UAV, it switches to

a collision avoidance mode. The collision avoidance mode of a UAV comprises the

22

following: (a) finding a UAV, from among the neighbors, which is most threatening,

and (b) performing a turn maneuver until the projected trajectories of the UAV and that

of the most threatening UAV do not conflict. At any instant of time, a UAV might have

predicted conflicts with several of its neighbors. The most threatening neighbor for a

UAV is the one with which the predicted near miss is the most immediate to occur.

The turn maneuver performed is one which ensures an increase in the Line-of-Sight

(LOS) rate between the concerned UAVs – a UAV and its most threatening neighbor.

This is inspired by and contrary to missile guidance principles where missile takes a

turn to decrease its LOS rate with the target (Zarchan, 1990) to effect a collision. The

collision avoidance maneuver would have taken the UAV out of its nominal path. It is

required that the arrival of a UAV at its destination has to be as quick as possible. The

UAV therefore takes a Dubins path (Dubins, 1957), which is a minimum time path,

from its current location to the destination.

We also extend this algorithm to three dimension (3D) which becomes readily

applicable to cases where engagements are inherently 3D. The two dimensional (2D)

collision avoidance algorithm proposed becomes a limiting case of this algorithm when

the engagement tends to be planar. Also proposed is a 3D algorithm which makes the

UAVs execute a 3D maneuver even for planar conflicts. The 3D maneuvers are ad-

vantageous in case of high density traffic, as the 3D deconfliction maneuver will take

UAVs to different altitude planes, thus reducing the possibility of subsequent conflicts.

This will become evident in the simulations presented later. The collision avoidance

mode in 3D consist of a UAV applying an acceleration in or out of its collision plane.

A collision plane for a UAV is defined as a plane that contains the UAV and the veloc-

ity vector of a UAV with which it has a predicted miss distance less than the required

separation and a minimum time-to-go. The application of acceleration in the colli-

sion plane leads to the 3D extension of the proposed 2D algorithm. The out of plane

application of acceleration pulls a UAV out of the collision plane resulting in a 3D col-

lision avoidance maneuver even for planar conflicts. In the 3D case also, a UAV heads

to its destination, after a deviation from its nominal path due to collision avoidance

maneuver, via Dubins path on a plane containing the UAV’s velocity vector and the

destination.

The central thesis of this chapter is that in a multiple UAV conflict scenario, ev-

ery UAV doing a maneuver to increase its LOS rate with a UAV with which it has the

most imminent projected near miss, results in a good collision avoidance performance.

Toward establishing this concept, we show, through simulations, that the resulting col-

lision avoidance algorithm handles pairwise conflicts correctly and every UAV using

23

this algorithm in a highly cluttered environment will result in the reduction of occur-

rence of near misses.

The collision avoidance algorithms that we develop in this chapter, implemented

by a UAV, use the position and velocity information of only those UAVs that are within

a UAV’s sensor range. The algorithms are decentralized, computationally less de-

manding, and require no communication among UAVs. They also perform well in the

presence of noise and disturbances, and are easy to implement.

2.2 Two dimensional collision avoidance algorithm

The Proportional Navigation (PN) guidance law (Guelman, 1971; Ghawghawe &

Ghose, 1996) is very robust and elegant, and its variants are therefore most widely

used for missile guidance. The PN guidance law applies a lateral acceleration to the

missile to nullify the rate of rotation of Line of Sight (LOS) or the LOS rate between

the missile and the target, and brings the missile into a collision course with the target.

For collision avoidance, a lateral acceleration which takes a vehicle away from the

collision course is what is desirable. We propose an algorithm that uses an ‘inverse’

PN for collision avoidance, in the sense that the algorithm consist of applying a lateral

acceleration to increase the LOS rate between UAVs that are on a collision course.

2.2.1 Conflict detection

The first step toward employing the proposed collision avoidance rule for a UAV is

to calculate the envisaged miss distance or the Zero Effort Miss (ZEM) with all other

UAVs which are in its sensor range. The sensor range of a UAV is assumed to be

defined by a circle of radius Rsen (sensor range radius) around the UAV. It is assumed

that every UAV has the capability to locate other UAVs in its sensor range and measure

their velocities.

In a 2D engagement scenario, consider two UAVs, U1 and U2. Let the current

positions and velocities (that is, at time t) of U1 and U2 be p1 = (x1,y1) and v1 =

(u1,v1), and p2 = (x2,y2) and v2 = (u2,v2), respectively. To keep the equations simple,

throughout this thesis unless otherwise specified, the current values of the quantities

that explicitly depends on time are represented by the variables denoting the quantities

themselves. For instance, the current position of Ui is represented as pi rather than

pi(t). At any future time t + τ, the new locations of U1 and U2 are p1 + v1τ and

p2 +v2τ , provided that the UAVs do not change their velocities. We consider collision

24

avoidance against UAVs with different but constant speeds, and use turn rates only

– no speed changes – for collision avoidance. This leaves the degree of freedom of

speed change to be used by other algorithms that co-exist with the collision avoidance

algorithm in a UAV. An example of this is later given in Chapter 6. If the separation

between the two UAVs at time t + τ is r, then

r = ‖(p1 +v1τ)− (p2 +v2τ)‖ (2.2)

where ‖ ‖ is the Euclidian norm and we get

r2 = (p1−p2)·(p1−p2)+2(p1−p2)·(v1−v2)τ +(v1−v2)·(v1−v2)τ2 (2.3)

Solving for τ from drdτ = 0 will give the time at which a minimum separation between

UAVs will occur. We call this the time-to-go (tgo) which is given as

tgo =−(p1−p2)·(v1−v2)(v1−v2)·(v1−v2)

(2.4)

If tgo > 0, then there is a time at which closest approach between two UAVs occur. The

tgo calculated as above can take negative values which means that the vehicles are on a

diverging path. That is, their paths extrapolated backward in time lead to a minimum

distance between them and thus a negative tgo. Substituting tgo for τ into the expression

for r in Eq. (2.2) will give the predicted miss distance or ZEM which is the distance of

closest approach if the UAVs proceed in their current course. Let �p = p1−p2, and

�v = v1−v2. Then

ZEM =

√‖�p‖2‖�v‖2−‖�p·�v‖2

‖�v‖2 (2.5)

In the computation of the predicted miss distance as above, it is assumed that the

heading of the UAVs do not change. Although this assumption is not strictly valid,

since the UAVs update the predicted miss distance at every instant using instantaneous

velocities, the near miss predicted using Eq. (2.5) in the course of time would approach

the ‘actual near miss’ which is obtained by taking into account the turn rates of the

UAVs. The approximate computation of predicted near miss is used in the algorithm

to keep the algorithm simple and to reduce the number of computations in view of

real time implementation. In principle, it is possible that the approximate near miss

computed as above would cause a UAV to execute unnecessary maneuvers that might

lead to reduced efficiency. The simulation results presented later show that the loss in

efficiency due to this reason, however, is less.

25

If the obtained miss distance is less than the desired separation, then a collision

avoidance maneuver has to be performed. A UAV makes a list of UAVs in its neighbor-

hood with which its miss distance is less than the minimum required separation. From

this list, the UAV chooses that UAV with which it has minimum positive tgo, which

means that they are on an immediate collision course, and does a collision avoidance

maneuver.

2.2.2 Collision avoidance maneuver

The collision avoidance maneuver that we propose involve UAVs pulling out of the

collision course. To pull out of a collision course, a UAV has to apply a lateral ac-

celeration in a way that will increase the rate of rotation of the LOS connecting the

two UAVs. This is illustrated using Fig. 2.1. Let r be the LOS separation. For the

engagement shown in Fig. 2.1, the LOS rate with respect to U1, is given as,

θ =V2 sinθ2−V1 sinθ1

r(2.6)

To increase the LOS rate in this scenario, accelerations a1 and a2 as follows should be

applied to U1 and U2, respectively, as shown in Fig. 2.1.

ai = sgn(θ)V 2

i

Ri, i = 1,2 (2.7)

where, sgn(θ) is +1 if the LOS rotates anticlockwise and −1 otherwise. Here, R1 and

R2 are the required radii of turn for U1 and U2. For a UAV with minimum radius of

turn Rmin, the required radius of turn is computed as

R = Rmin exp

(λ ×ZEM

Rdes

)(2.8)

which is an empirical relation that we propose. Here, the desired separation Rdes and

λ are tuning parameters for the algorithm. The desired separation, Rdes, takes a value

greater than the required minimum separation. Appropriate values for Rdes and λ are

obtained through Monte Carlo simulations. The choice of these tuning parameters is

a trade-off between the efficiency and the number of near misses. Section 2.5 gives a

study of the performance of the proposed collision avoidance algorithm as these tuning

parameters vary.

The direction of application of the lateral acceleration is such that the LOS rate

between the UAVs would increase. There are only two directions perpendicular to the

velocity of a UAV along which a lateral acceleration can be applied, one of which will

bring the UAV closer to collision course while the application in the other direction

26

V1

V2

a1

a2

θ1

θ2

rLOS

U1

U2

θ(x1,y1)

(x2,y2)

x

y

Figure 2.1: A 2D engagement between two UAVs.

Algorithm 2.1: Deconfliction algorithm used by UAV Ui

Find neighbors Ni that are other UAVs within the sensor range Rsen of Ui.

Calculate ZEM between Ui and each UAV Uj ∈Ni.

Find N ∗i , the neighbors with which Ui has ZEM < Rdes.

if N ∗i is not empty then

Choose Uj∗ ∈N ∗i with which Ui has least tgo.

Turn with radius of turn R to increase LOS rate between Ui and Uj∗.

end if

will cause an increase in the LOS rate thus aiding the UAV to pull out of the collision

course.

The collision avoidance algorithm proposed for a multiple UAV conflict scenario

is given in Algorithm 2.1.

The proposed collision avoidance algorithm has the effect of increasing the LOS

rate between two UAVs in a collision course. As mentioned earlier, this is exactly

opposite to the PN philosophy that applies a lateral acceleration proportional to the

LOS rate between the missile and the target, and tries to nullify the LOS rate. A

simplistic implementation of a PN based avoidance law would be to apply a lateral

acceleration inversely proportional to the LOS rate. In Eq. (2.8), the turn radius is

a strictly monotonously increasing function of ZEM. When the ZEM is zero, a UAV

executes a collision avoidance maneuver with the tightest turn. Whereas, for a higher

27

ZEM, as there is a lower risk of collision or near miss, the turn rate of the collision

avoidance maneuver of a UAV is considerably less, and thus R is higher, to minimize

deviation from the nominal path. Note that, in the algorithm, although the philosophy

of increasing the LOS rate is used, the maneuver itself does not use the LOS rate in its

expression.

The proposed multiple UAV collision avoidance algorithm operates by deconflict-

ing the most imminent pairwise collisions. However, this control action may lead a

UAV into a collision course with another UAV. Nonetheless, the central thesis of this

chapter holds that, deconflicting critical pairwise conflicts suffices to address multiple

UAV collision avoidance. Although there is no guarantee that all the conflicts will be

eventually resolved, the extensive simulations presented later reassures that it is most

likely the case. Since the proposed algorithm relies on deconflicting the most critical

conflict, it is important to show that the proposed algorithm handles pairwise con-

flicts correctly. In the following subsection, the effectiveness of the proposed collision

avoidance algorithm in handling pairwise conflict is studied.

2.2.3 Effectiveness of handling pairwise conflicts

Consider an engagement of two UAVs as in Fig. 2.1. As seen in the figure, the UAVs

U1 and U2 have an engagement geometry, determined by θ1 and θ2, and an initial

separation r. If these UAVs execute a collision avoidance maneuver by taking a turn

of constant radius R in the directions (along a1 and a2) as indicated in the figure,

then the LOS separation between them initially decreases, reaches a minimum and

then increases. The evolution of the LOS separations between U1 and U2 in time,

when UAVs execute a tightest radius of turn Rmin, is shown in Fig. 2.2 (solid curves)

for different engagement geometries (θ1 and θ2), constant velocity of V1 = V2 = V =

500 miles per hour (223.5 m/s), and an initial separation of 10 miles. These values,

that are typical of a large aircraft rather than a UAV, were chosen as we, in a later

section, compare the results obtained using our algorithm with that of an algorithm in

the literature (Archibald et al., 2008) which considers deconfliction of large aircraft in

high density traffic environments. The Air Traffic Control (ATC) regulations require

that the desired separation between two aircraft at any point of time in flight be greater

than 5 miles. As the ATC regulations are specified in miles, we, in this chapter, have

used miles as the standard unit. However, in the next chapter, where we demonstrate

the efficacy of our algorithm implemented on realistic UAV models, we use the data

corresponding to small UAVs whose dimensions are in the order of meters, as it is

28

0 5 10 15 20 256.5

7

7.5

8

8.5

9

9.5

10

time (sec)

min

imum

sep

arat

ion

(mile

s)

R = Rmin

R as in (2.8)

θ1=15°, θ

2=45°

θ1=0°, θ

2=45°

θ1=0°, θ

2=0°

Figure 2.2: Variation of the LOS separations between two UAVs with time for a turnradii R = Rmin and R as in Eq. (2.8) for various initial engagement geometries.

envisaged that small UAVs rather than large aircraft are more likely to be used in high

density traffic operations.

We make the following observations from the simulations, as described above, of

pairwise engagements of UAVs. From Fig. 2.2, we observe that the most critical en-

gagement is head-on (that is, θ1 = 0 and θ2 = 0). Also, we note that the minimum

separation and the time at which it occurs depends on the engagement geometry. Be-

low, we derive a closed form expression for the time at which minimum separation

occurs from which we will see that the minimum separation and the time at which it

occurs not only depends on θ1 and θ2 but also on initial separation r and the turn radius

R.

The maneuver as described above involves, as shown in Fig. 2.3, the UAVs U1 and

U2 taking turns along circles C1 and C2 of radius R each. Without loss of generality,

we can assume that the UAVs U1 and U2 at any time t are at (0,0) and (r,0). For this

case, the center (xc1 ,yc1) of circle C1 is given as

xc1 = −Rsinθ1

yc1 = Rcosθ1 (2.9)

The center (xc2 ,yc2) of circle C2 is

xc2 = r−Rsinθ2

yc2 = −Rcosθ2 (2.10)

29

(xc2,yc2

)

(x2,y2)

(xc1,yc1

) (x1,y1)

R

R

rDj

DjU1

U2q1 q2

C1

C2

V

V

x

y

Figure 2.3: Collision avoidance maneuver for a 2D engagement.

After a time Δt, the heading change for each UAV is Δψ = ψΔt = (V/R)Δt. Let

(x1, y1) be the position of U1 after Δt time. From Fig. 2.3, we get

x1 = xc1 +Rcos(

Δψ−(π

2−θ1

))= xc1 +Rsin(Δψ +θ1)

y1 = yc1 +Rsin(

Δψ−(π

2−θ1

))= yc1−Rcos(Δψ +θ1) (2.11)

Similarly, let (x2, y2) be the position of U2 after Δt time. Then

x2 = xc2−Rsin(Δψ−θ2)

y2 = yc2 +Rcos(Δψ−θ2) (2.12)

Thus, the separation between U1 and U2 at Δt is given as

r =√

(x1− x2)2 +(y1− y2)

2 (2.13)

We differentiate r with respect to Δψ and set the resultant to zero to find that value of

heading change Δψ at which the minimum separation occurs. This is obtained as

Δψmin = arctan

[r(cosθ1 + cosθ2)

2R{1+ cos(θ1 +θ2)}+ r(sinθ2− sinθ1)

](2.14)

Note that the heading change obtained depends on the initial engagement parameters

(θ1,θ2,r) and the radius of turn R. If V is the magnitude of velocity of UAVs, then

the heading rate during a turn of radius R is ψ = V/R. If the minimum separation

between the UAVs occurs after a time Δt from the time of initial separation, then

Δψ = (V/R)Δt. Therefore, for a given speed and radius of turn, the time Δt after

30

−100 −50 0 50 1006.5

7

7.5

8

8.5

9

9.5

10

θ2 in degrees

min

imum

sep

arat

ion

(mile

s)

R = Rmin

R as in (2.8)

θ1 = −45°

θ1 = 45°

θ1 = 0°

θ1 = 90°θ

1 = −90°

Figure 2.4: The minimum LOS separations attained between two UAVs for radii ofturn Rmin and R as in Eq. (2.8) (various initial engagement geometries).

which a minimum separation occurs depends on θ1 and θ2, since Δt, which is given

as (R/V )Δψ , depends on Δψ . This is evident from Fig. 2.2. For different initial

engagements, the evolution of the LOS separations between the UAVs with time when

the UAVs use maneuvers that depend on the predicted miss distance (that is, radius of

turn R used is as in Eq. (2.8)) are shown in Fig. 2.2 as dotted curves. Although the

effort is less in the case where R in Eq. (2.8) is used, the minimum separation achieved

is also less but is still more than the required separation.

Figure 2.4 gives a plot of how the minimum separation between two UAVs varies

with θ2 for different values of θ1. Cases where θ1 = {−90,−45,0,45,90} degrees are

plotted in the figure. The minimum separation plotted is the closest approach between

UAVs U1 and U2, when the initial engagement geometry is (θ1,θ2), and the UAVs

execute a maneuver of tightest turn with radius Rmin. It is observed that the critical

case is one where θ1 = θ2 = 0◦, that is, a head-on approach. In the algorithm of this

paper, however, instead of the tightest turn maneuver an instantaneous turn radius R, as

given in Eq. (2.8), is used which depends on the predicted miss distance between the

UAVs at each instant. The minimum separation between two UAVs for different initial

geometries for this case is also given in Fig. 2.4. The figure shows that the minimum

separation is less than that in the case where maneuvers involve the tightest turn but is

well above the required minimum separation of 5 miles even for the case of head-on

approach.

31

Algorithm 2.2: Algorithm to implement Dubins path with free terminal constraints

if UAV heading points to D then

Head straight.

else if C+ encircles D (D strictly within C+) then

Take a path along C−.

else

Take a path along C+.

end if

(refer Fig. 2.5 for definitions of D , C+, and C−)

2.2.4 Dubins path to destination

Given initial position and departure angle, and final position and arrival angle, Du-

bins curve or path gives the trajectory with minimum path length under the additional

constraint of a maximum allowed curvature of the trajectory (Dubins, 1957). In the

multiple UAV collision avoidance scenario, once a UAV has deviated from its nom-

inal path after a collision avoidance maneuver, it is desirable to reach its destination

in minimum time. Following a Dubins path to destination will help in achieving this

because, with a constant speed assumption, a minimum length path is also a minimum

time path. If every UAV chooses a Dubins path (minimum length path) from their cur-

rent locations to their respective destinations, it provides an easy method to measure

the deviation due to collision avoidance maneuvers in terms of the delay in the arrival

of a UAV at its destination. For arrival of a UAV at destination in minimum time, a

special case of Dubins curve is required where there is no arrival angle constraint. The

maximum curvature constraint takes care of the minimum radius of turn of the UAV. A

switching algorithm is proposed as follows that implements this special case of Dubins

path.

Let D be the destination, C+ a circle, whose radius is equal to the radius of min-

imum turn, tangential to the velocity vector and such that following C+ will take the

UAV toward the destination, and C− a similar circle such that following C− will take a

UAV away from the destination (refer Fig. 2.5). Then, a UAV employing Algorithm 2.2

32

Destination

C+

C- D

(a) Case were destination is outside the circle (C+)of minimum radius of turn

DestinationC+

C-

D

C+

(b) Case were destination is within the circle(C+) of minimum radius of turn

Figure 2.5: Dubins paths to destination – two possible scenarios (the case of freeterminal constraint).

will follow a Dubins path to the target. Figure 2.5 shows the possible cases of Dubins

path to a destination when there is no terminal angle constraint. Figure 2.5(a) gives

Dubins path to destination in a case where the destination is far from the UAV and

Fig. 2.5(b) illustrates the case where the destination is too close (within C+).

This algorithm is implemented with a slight modification. During simulation, a

UAV follows Algorithm 2.2 till the difference between its desired heading, in which

case the UAV’s velocity vector points directly to its destination, and actual heading is

less than a small quantity ε > 0. Beyond that, the UAV uses a proportional controller

of the form ψ = k(ψd−ψ), where ψ is the heading angle and ψd is the desired heading

of the UAV. This avoids chattering that might occur otherwise due to switching.

2.2.5 RCA-2D: two dimensional collision avoidance algorithm

Combining the algorithms for conflict resolution and heading to destination maneu-

vers, we obtain the collision avoidance algorithm, that each UAV flying to its destina-

tion implements at every time step in a multi-UAV scenario, in Algorithm 2.3. Since

a UAV reacts to the situation of a violated desired separation this 2D algorithm is

referred to as Reactive Collision Avoidance (RCA-2D) Algorithm in this thesis.

RCA-2D handles the multiple UAV scenario by considering only that collision

which is immediate to a UAV in terms of the minimum tgo. This approach results in

a good collision avoidance algorithm with good performance as we will demonstrate

through simulations. However, it is possible that by considering only the immediate

collision and doing a collision avoidance maneuver may at times lead the UAVs to

other conflicts which are too close to avoid. Also, there is a chance of chattering

33

Algorithm 2.3: Reactive Collision Avoidance 2D Algorithm (RCA-2D)

Use Algorithm 2.1 detect and avoid near misses.

if there are no predicted near misses then

Use Algorithm 2.2 for homing to destination.

end if

when a UAV faces two nearly similar conflicts – avoiding one will lead to the other

and vice versa. This usually occurs when engagement geometries have some sort of

symmetry. In such cases where engagements are close to symmetric, it is desirable to

follow a protocol which will break the symmetry and then proceed to use RCA-2D.

We remark that this algorithm can easily handle heterogeneous group of UAVs with

different velocities and radii of turn by using tuning parameters specific to individual

UAVs instead of global ones. However, to keep the situation simple, we consider only

a homogeneous group of aircraft for simulations presented in this chapter. Simulations

involving heterogeneous UAVs are presented in Chapter 3.

2.3 Three dimensional conflict avoidance

The 2D collision avoidance algorithm, developed in the last section, is simple in con-

cept and implementation. However, in high density air traffic scenarios, the number

of near misses and the deviations of UAVs from nominal paths to their respective des-

tinations due to collision avoidance maneuvers restricted to 2D may still be higher

than tolerable levels. Hence, it seems reasonable to take advantage of the unused third

dimension rather than trying to improve the 2D algorithm that may result in compli-

cated and computationally demanding solutions. Also, if the initial engagement (that

is, when UAVs appear in each other’s sensor range for the first time) is non-planar,

which probably may be the case most of the time, then a 2D deconfliction algorithm is

not helpful. A sample situation in which the conflicts among UAVs are in 3D is shown

in Fig. 2.6. The inclusion of 3D maneuvers into the set of possible collision avoidance

maneuver candidates greatly improves the freedom to maneuver. We devise a 3D col-

lision avoidance algorithm using similar ideas as in the previous section. Even in the

3D case, the key idea will be to increase the rate of rotation of the LOS, in the spirit

34

−100−50

050

100

−100

−50

0

50

−40

−20

0

20

x−range

y−range

heig

ht

Figure 2.6: A sample situation requiring 3D collision avoidance among multipleUAVs.

of inverse PN, to facilitate collision avoidance. We propose a 3D collision avoidance

algorithm as follows.

2.3.1 Collision prediction

As in the 2D case, a UAV computes the predicted miss distances between all of its

neighbors and itself, and if it is less than the required separation, it does a collision

avoidance maneuver. The predicted miss distances are calculated exactly as in the 2D

case except that the vectors have an additional component that corresponds to the third

dimension. Let U1 and U2 be two UAVs located at p1 = (x1,y1,z1) and p2 = (x2,y2,z2)

with velocity vectors v1 = (u1,v1,w1) and v2 = (u2,v2,w2) of magnitude V at time t.

At time t + τ, if r is the separation between these UAVs, then

r = ‖(p1 +v1τ)− (p2 +v2τ)‖ (2.15)

The closest approach occurs at a time tgo obtained by solving for τ from dr/dτ = 0.

This gives an expression for tgo as

tgo =−(p1−p2)·(v1−v2)(v1−v2)·(v1−v2)

(2.16)

These equations are exactly the same as Eqs. (2.2) and (2.4) in the 2D case. If tgo >

0, the separation between the UAVs will reduce over time until tgo at which time a

minimum separation occurs. The value of minimum separation can be obtained by

plugging in tgo for τ in the expression for r. An expression same as Eq. (2.5) is obtained

35

for the zero effort miss. If the calculated minimum separation is less than the desired

separation, then a collision avoidance maneuver needs to be performed.

Central to the 3D collision avoidance algorithms is the idea of a ‘collision plane’

which is defined as a two dimensional plane in 3D on which if a UAV stays will re-

sult in a collision with another UAV, provided that both the UAVs do not alter their

directions. In the 3D extension of the 2D collision avoidance algorithm, the velocity

of a UAV is projected onto its collision plane and then the 2D algorithm is applied on

the collision plane to obtain a deconfliction. This reactive 3D collision avoidance al-

gorithm in which the acceleration for collision avoidance maneuver is on the collision

plane is referred to as RCA-3D-I, where ‘I’ stands for the ‘maneuver in the collision

plane’. In the purely three dimensional collision avoidance algorithm, the idea of colli-

sion avoidance by pulling out of collision course in 2D is incorporated in 3D as pulling

out of the collision plane. This algorithm will be referred to as RCA-3D-O, where ‘O’

is for the ‘maneuver out of the collision plane’. The choice of a collision plane from

among the infinite number of possible planes in 3D and the rationale for doing so is

explained in the next subsection.

2.3.2 Collision avoidance maneuver

In 2D, there is a unique way to do the PN inspired collision avoidance. In an engage-

ment in two dimensional plane, there are only two directions in which an acceleration

perpendicular to velocity can be applied – application of acceleration in one direction

will take the vehicle toward the collision triangle while the other will pull the vehicle

out of the collision course. In 3D, there is an infinite choice of directions, all of which

lie on a plane perpendicular to the velocity vector of the UAV, along which an acceler-

ation can be applied to increase the LOS rate and thus effect collision avoidance. The

main idea behind the design of a 3D algorithm for collision avoidance is to choose a

direction, out of these infinite ones, along which an acceleration has to be applied to

achieve a collision avoidance. The direction in which acceleration is to be applied is

chosen as the one which will pull the UAV out of the collision plane. The rationale for

the specific choice of collision plane and the acceleration direction is as follows.

Analysis of guidance laws has shown that they exhibit a time scale behavior (Naidu

& Calise, 2001). It has been recently shown that, in the 2D case, better guidance laws

have larger time scale gap (Dhananjay et al., 2010) which essentially means that a good

guidance law will quickly bring the missile on a collision course on which it stays for

the rest of the engagement. Similarly, it is expected that a good 3D guidance law will

36

U2

v2

U1

v1

(a) Three dimensional engagement

(b) Collision plane

Figure 2.7: Engagement scenario and collision plane in a 3D conflict.

37

take a missile quickly to a ‘collision plane’ on which a 2D-like engagement follows.

It looks reasonable, at least in the case of a non-stationary non-maneuvering target, to

define this collision plane as the plane containing the missile, the target and the target’s

velocity vector. This is because, being on this plane is a necessary condition for the

missile to hit the target.

The development of RCA-3D-O is explained below. As in 2D where the appli-

cation of 2D collision avoidance algorithm to a UAV results in its pulling out of a

collision course, it is desirable for a 3D algorithm to drive a UAV out of the colli-

sion plane. Below, an example is given with illustration of the collision plane and the

direction in which an acceleration should be applied to effect a deconfliction in 3D.

Figure 2.7(a) shows a 3D engagement scenario. As it follows from the previous

discussion, the collision plane for a UAV U1 which is on a collision course with another

UAV U2 is a two dimensional plane containing both U1 and U2, and the velocity vector

of U2. Figure 2.7(b) illustrates this which shows the collision plane for U1. To pull

out of the collision plane, the UAV U1 ideally needs to apply an acceleration in a

direction n1 perpendicular to the collision plane as shown in Fig. 2.7(b). The vector

n1, perpendicular to the collision plane, is chosen such that n1 ·v1 > 0. However, this

may not be perpendicular to the velocity vector of the UAV. Therefore, acceleration

is applied along a1out which lies in the plane of v1 and n1 (refer Fig. 2.7(b)). Also,

a1out is perpendicular to v1 and a1out·n1 > 0. The magnitude of a1out is V 2/R where

R = Rmin exp(λ ×ZEM/Rdes). Here Rmin is minimum turn radius of U1, ZEM is the

predicted minimum separation between U1 and U2, and the desired separation Rdes and

λ are tuning parameters for the algorithm.

Having defined the collision plane, it is possible to extend the 2D algorithm in

Section 2.2 to handle three dimensional conflicts. The existence of a predicted near

miss for a UAV would imply that its velocity vector is close to the collision plane. The

3D algorithm proposed above increases the LOS rate by pulling the velocity vector of

the UAV away from the collision plane. Another way to handle a three dimensional

conflict will be to project the velocity vector of the UAV on to the collision plane and

apply the lateral acceleration to increase the LOS rate in the collision plane. This

algorithm, RCA-3D-I, is a natural extension of the proposed 2D algorithm to handle

inherently three dimensional conflicts. Let v1c be the projection of v1 on the collision

plane. Now, the 2D algorithm proposed in the previous section is used on the collision

plane assuming that U1 has a velocity v1c. The acceleration obtained a1in is in the

collision plane and is perpendicular to both v1c and v1 (see Fig. 2.7(b)). Similar to

RCA-3D-O, a1in has a magnitude equal to V 2/R.

38

RCA-3D-I and RCA-3D-O are, in some sense, at two different extremes – one

proposes a maneuver in the collision plane while the other perpendicular to it. A linear

combination of these proposed accelerations in the form a = λain +(1−λ )aout ,λ ∈[0,1], can be considered for the three dimensional collision avoidance. However, we

do not follow that approach in this thesis, but study only the ‘extreme cases’, that is,

the cases where λ is either 0 or 1.

2.3.3 Homing to destination in three dimension

Execution of a collision avoidance maneuver may take a UAV out of its nominal path

to the destination. When there are no more threats of collisions or near misses, the

UAV has to choose and follow a trajectory which will take it to its destination without

much further delay. We propose an algorithm to do this in the 3D case as follows.

Given the current location of UAV and its destination in a 3D space, a plane con-

taining the UAV, the destination, and the current velocity of the UAV can be uniquely

found. A Dubins path is followed on this plane to reach the destination. After finding

the plane containing the velocity vector and destination of a UAV, the UAV follows a

Dubins path on this plane in exactly the same manner as in 2D using Algorithm 2.2.

(Details of implementation of this is given in step 7 in Section 2.4.2 which describes

the implementation of the 3D collision avoidance algorithm.)

2.4 Implementation of collision avoidance algorithms

To evaluate the collision avoidance algorithm developed, several simulations are car-

ried out. This section gives the implementation details of algorithms RCA-2D, RCA-

3D-O, and RCA-3D-I as well as that of Satisficing Game Theory (SGT) based algo-

rithm (Archibald et al., 2008), which is used for comparative studies in section (Sec-

tion 2.5) through simulation results.

2.4.1 Implementation of the RCA-2D algorithm

Consider a general scenario where N number of UAVs fly from their origins to respec-

tive destinations. The algorithm RCA-2D consists of each UAV Ui ∈U = {U1,U2, . . . ,

UN} executing the following six steps at each time step.

Step 1: Find the neighbors within the sensor range. The set of neighbors for a UAV Ui

is

Ni = {Uj ∈U, j = i : ‖pi−p j‖ ≤ Rsen}

39

Step 2: If Ni is empty proceed to Step 5, otherwise, from among the neighbors, find

those with which Ui has a Zero Effort Miss (ZEM) less than Rdes. To do this, calculate

ZEM of Ui with a UAV Uj ∈Ni, ZEM(Uj), as in Eq. (2.2). The set of neighbors with

which Ui has a predicted miss distance less than the required separation Rdes is

N ∗i = {Uj ∈Ni : ZEM(Uj)≤ Rdes}

The UAVs Ui and Uj have velocities vi and v j. Rdes is a tuning parameter of the

algorithm.

Step 3: If N ∗i is empty (that is, no neighboring UAVs are on a collision course with

Ui) proceed to Step 5. Otherwise, find the neighbor with which it has a predicted

miss distance less than the desired separation and the least time-to-go for that violated

separation to occur as

Uj∗ = arg minUj∈N ∗

i

tgo(Uj)

where, tgo(Uj) is the time-to-go for minimum separation between Ui and Uj and can

be calculated using Eq. (2.4).

Step 4: Find the heading update for UAV Ui to avoid Uj∗ as follows. Let ψi and ψ j∗be the headings of Ui and Uj∗, respectively and V be their constant speed. Let θ be

the LOS angle and r the LOS separation as indicated in Fig. 2.1. Then the LOS rate is

given as

θ =V sin(ψ j∗ −θ)−V sin(ψi−θ)

r

Obtain the required heading as

Δψi =

⎧⎪⎪⎪⎨⎪⎪⎪⎩−Vi

RΔt , if θ ≥ 0

+Vi

RΔt , otherwise

where, R = Rmin exp(λ ×ZEM(Uj∗)/Rdes).

Here, ZEM(Uj∗) is the predicted minimum separation between Ui and Uj∗ as com-

puted using Eq. (2.2), Rmin is the minimum radius of turn, λ is a tuning parameter of

the algorithm, and Δt is the time step. After obtaining the required heading change,

proceed to Step 6.

Step 5: If a UAV is not on a collision course with any other UAVs then there is no need

for a collision avoidance maneuver. So proceed to find the heading update to follow a

40

Dubins path to destination. Let pi = (xi,yi,0) be position of UAV Ui and pt = (xt ,yt ,0)

be its destination. Also, let vi = (ui,vi,0) be the UAV velocity and ψi be its heading.

As the first step toward implementing the Dubins path to destination, find the center

(xc,yc) of the circle C+ (refer Fig. 2.5) toward the target and the turn direction Dψ as

⎛⎝ xc

yc

Dψ

⎞⎠ =

⎧⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎩

⎛⎝ xi−Rmin sinψi

yi +Rmin cosψi

+1

⎞⎠ , if sgnz (vi×(pt−pi)) > 0

⎛⎝ xi +Rmin sinψi

yi−Rmin cosψi

−1

⎞⎠ , otherwise

Here, Dψ indicates the direction in which the UAV has to turn to follow C+. A value

of +1 for Dψ represents an anticlockwise turn and −1 a clockwise turn. We define the

function sgnz() to give the sign of z component of that vector which is the argument of

the function.

The heading change for a unit time step that takes the UAV to its destination (xt ,yt)

via the Dubins path is obtained as

Δψi =

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩

+Dψ

(V

Rmin

)Δt , if ‖(xc,yc)− (xt ,yt)‖> Rmin

−Dψ

(V

Rmin

)Δt , otherwise

If the desired heading is very close to the actual heading, we use a proportional control

to avoid chattering due to frequent switching of computed values of Δψi. That is, if

|ψd −ψi| < ε then Δψi = k(ψd −ψi)Δt. For simulations k = 1 and ε = 10−6 radians

are used.

The desired heading can be found as

ψd = arctan

[(pt−pi) · j(pt−pi) · i

]

where, i and j are unit vectors along x and y coordinates, respectively.

Step 6: Update the heading and position of Ui as

ψi = ψi +Δψi , xi = xi +Vi cosψiΔt , yi = yi +Vi sinψiΔt

The RCA-2D algorithm consist of every UAV Ui ∈U implementing these steps at

every time step until they reach the respective destinations.

41

2.4.2 Implementation of the RCA-3D-O algorithm

The 3D algorithm, RCA-3D-O, is implemented by each UAV Ui,1≤ i≤N, as follows.

Step 1: Find the neighbors, Ni as in Step 1 of RCA-2D algorithm.

Step 2: If Ni is not empty, find N ∗i ⊆Ni from among the neighbors, with which there

is possibility of near miss as in Step 2 of RCA-2D. Otherwise proceed to Step 7.

Step 3: If there are no neighbors with which there is a possibility of near miss then

proceed to Step 7. Otherwise find the neighbor Uj∗ with which it has minimum tgo for

near miss to occur as in Step 3 of RCA-2D.

Step 4: Find a vector perpendicular to the collision plane of Ui as follows. Let the

positions of UAVs Ui and Uj∗ be pi and p j∗, respectively and their velocities be vi and

v j∗. Collision plane is the plane containing UAVs Ui and Uj∗, and the velocity vector

of Uj∗. Therefore, we can find a vector perpendicular to the collision plane as

ni = v j∗×(p j∗ −pi)

Step 5: Find a component of ni perpendicular to vi, which is the direction in which

acceleration has to be applied for collision avoidance, as

ai = ni− (ni · vi)vi

where ‘ˆ’ represents the unit vector along the original vector direction.

Step 6: Find the required change in velocity vector for unit time step as

Δvi =

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩

+ai

(V 2

R

)Δt , if ni ·vi > 0

−ai

(V 2

R

)Δt , otherwise

Proceed to Step 8 for position and velocity update.

Step 7: Follow the Dubins path to destination if there is no need for collision avoidance

(that is, either Ni or N ∗i is empty).

The Dubins path in 3D lies on a plane containing velocity vector of UAV and its

destination point. So, it can be reduced to a 2D problem in a new coordinate system

where the x axis is along d = pt −pi, the UAV is at the origin, and the velocity vector

makes an angle ψi with x axis. The heading of UAV in this new axis, ψi, is

ψi = arccos

(vi · (pt−pi)‖vi‖‖pt −pi‖

)

42

The center of C+ circle in Fig. 2.5 in new coordinates is

xc = Rmini sin ψi , yc =−Rmini cos ψi

The acceleration direction perpendicular to the velocity vi in the original coordinates

is

d⊥ = d− (d · vi)d

Find the velocity change for a unit time step to steer the UAV toward destination along

Dubins path in 3D is

Δvi =

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩

d⊥(

V 2

Rmin

)Δt , if ‖(xc,yc−‖pt−pi‖)‖> Rmin

−d⊥(

V 2

Rmin

)Δt , otherwise

Step 8: Update velocity and position of UAV Ui as

vi = vi +Δvi , pi = pi +viΔt

2.4.3 Implementation of the RCA-3D-I algorithm

The 3D algorithm RCA-3D-I implemented by each UAV Ui,1≤ i≤ N, is as follows.

Step 1: Find the neighbors, Ni as in Step 1 of RCA-2D algorithm.

Step 2: If Ni is not empty, find from among the neighbors the ones, N ∗i , with which

there is a possibility of near miss as in Step 2 of RCA-2D. Otherwise proceed to Step 9.

Step 3: Find the neighbor Uj∗ with which it has minimum tgo for near miss to occur as

in Step 3 of RCA-2D.

Step 4: Find a vector, ni, perpendicular to the collision plane as in Step 4 of RCA-3D-

O.

Step 5: Find the component of vi in the collision plane as

vic = vi− (ni ·vi)ni

where ‘ ˆ ’ represents the unit vector along the original vector direction. Now, vic and

v j∗ are on the same plane which is the collision plane.

Step 6: Find the component of LOS rate in the collision plane, θi j∗c, as follows. Let

Δp∗ = p j∗ −pi. Then,

θi j∗c =[v j∗ − (v j∗·Δp∗)Δp∗]− [vic− (vic·Δp∗)Δp∗]

‖Δp∗‖

43

Step 7: Find a component of ni perpendicular to vi, which is the direction in which

the acceleration has to be applied for collision avoidance, as

ai = ni− (ni · vi)vi

where ‘ˆ’ represents the unit vector along the original vector direction.

Step 8: Find required change in velocity vector for unit time step as

aiin =

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩

+vic×ni

(V 2

i

Ri

)Δt , if θi j∗c < 0

−vic×ni

(V 2

i

Ri

)Δt , otherwise

Proceed to Step 10 for position and velocity update.

Step 9: Follow Dubins path to destination if there is no need for collision avoidance

(that is, either Ni or N ∗i is empty) as in Step 7 of RCA-3D-O.

Step 10: Update velocity and position of UAV Ui as in Step 8 of RCA-3D-O.

2.4.4 Aircraft conflict resolution using Satisficing Game Theory (SGT)

The main contribution of this chapter of the thesis is the development of a simple,

computationally efficient and easily implementable algorithm for collision avoidance

among multiple UAVs. Simulations in this chapter consider scenarios where the traffic

densities are very high. Since very few algorithms exist in the literature addressing this

problem, the satisficing game theory based algorithm proposed recently as a solution to

this problem by Hill et al. (2005) and Archibald et al. (2008) is chosen for comparison

with the results of RCA-2D and RCA-3D. As complete details of the implementation

in (Hill et al., 2005) and (Archibald et al., 2008) is not available in these papers, the

implementation of the SGT algorithm as used for simulation results presented in this

chapter is described below. The results used for comparison are obtained from this

implementation of the SGT algorithm and not directly taken from the references (Hill

et al., 2005) and (Archibald et al., 2008). However, the results obtained in the noise-

free case are similar to the ones reported in (Hill et al., 2005) and (Archibald et al.,

2008). These papers do not consider the noisy case.

Now we give a brief description of the SGT for which we mostly follow the no-

tations in (Hill et al., 2005; Archibald et al., 2008). In the satisficing game theoretic

approach, the preferences or the lack of it of a player/agent over decisions are repre-

sented using utility functions called selectability and rejectability, respectively. These

44

are maps from the space of decision variables to [0,1] and have properties that resem-

ble that of a probability distribution function. If PS and PR are the selectability and

rejectability functions and U = {u1,u2, . . . ,um} is the set of possible actions, then

PS : U −→ [0,1] , and

PR : U −→ [0,1]

with properties

PX(u)≥ 0 ∀u ∈U , and ∑u∈U

PX = 1 for X = S,R.

The construction of PS and PR is described below in Steps 3-6. The decision of a player

is

u∗ = argmaxu∈U

PS(u)−PR(u)

The key idea in applying satisficing game theory to any problem is to formulate mean-

ingful selectability and rejectability functions. In the aircraft collision avoidance prob-

lem considered, the decision of each aircraft is the choice of heading. Thus the decision

set is

U = {turn 5◦ left, turn 2.5◦ left, fly straight, turn 2.5◦ right, turn 5◦ right}

If the time step used for simulations, Δt, is 1 second, then the above decision set cor-

responds to a maximum turn rate of 5◦ per second. The choice of five discrete heading

decisions greatly reduces the complexity of computations as the SGT algorithm is

greatly dependent on the size of the decision set.

The SGT algorithm for collision avoidance consists of each UAV Ui ∈U executing

the following steps at each time step.

Step 1: Find the neighboring UAVs, Ni, as in Step 1 of RCA-2D.

Step 2: Create a priority list Pi of the UAVs in Ni as follows. All UAVs within

5 miles of respective destination belongs to one group which has priority over all the

remaining UAVs which belong to the second group. Within each group, UAVs with

higher estimated delay in arriving at the destination has higher priority. Among the

UAVs with same delay, the one with higher flight time has higher priority. All the

UAVs in Ni with higher priority than Ui forms the set Pi.

The delay of a UAV in arriving at the destination is estimated as follows. The Du-

bins path length, dDubins, to target from the current location of a UAV can be calculated

at every time. Then its delay in arriving at the destination is

delay = t +dDubins

V− (tinit + texpected)

45

where, t is the current time (the time elapsed since the starting of simulation), V is the

constant speed of UAV, tinit is the time at which UAV was introduced, and texpected is

the time taken by the UAV to take a Dubins path from origin to destination if there are

no collision avoidance maneuvers. The flight time is calculated as t− tinit .

Step 3: Compute the rejectability function for Ui as follows.

PRi(u) ∝ ∑Uj∈Pi

WR(Ui,Uj), u ∈U

for a heading choice u of Ui and current heading of Uj where, WR is given as

WR(Ui,Uj) =

⎧⎨⎩

2α , if dmin(i, j)≤ Rc

α , if Rc ≤ dmin(i, j)≤ Rnm

0, otherwise

where,

α =

⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩

[1+

Rnm−dmin(i, j)Rnm

][1

d(i, j)

]β, if d(i, j)≤ 3Rnm

[1

d(i, j)

]β, otherwise

where, Rnm is the minimum separation required to avoid near miss, Rc is the separation

required to avoid physical collision, and β is a tuning parameter. Following Archibald

et al. (2008), we use a value of β = 2/3. The zero effort miss distance dmin(i, j)

between Ui and Uj, and the time-to-go, tgo, for that to occur, are calculated using

Eqs. (2.2) and (2.4). The distance from current point to point of closest approach

between Ui and Uj is d(i, j) and is calculated as V × tgo. The rejectability function is

then normalized so that ∑u∈U PRi(u) = 1.

Next step is the calculation of selectability function. For this, the simplified model

given in (Hill et al., 2005) and (Archibald et al., 2008) is used.

The selectability function of each player is influenced by preferences of other play-

ers who are higher in the priority list. The idea is that a player is ready to sacrifice his

efficiency in terms of directly heading to target if he is sure that another player higher

in the priority list will take advantage of his deviation.

Step 4: The first step in calculating the selectability map is to build the base selectabil-

ity. The base selectability of a choice of heading is more if it takes the aircraft closer

to target.

Each heading decision u, is assigned a rank r(u) depending on how close it takes

the UAV to its target. Since there are five heading decision choices, the decisions are

46

ranked from 1 to 5. Let u be the heading that takes the UAV directly to the target. The

base selectability map σSi for Ui is constructed as follows

σSi(u) ∝

⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩

3, r(u) = 12, r(u) = 22, r(u) = 3, 2.5◦ < |u−u| ≤ 5◦

1.1, r(u) = 3, 5◦ < |u−u|1.1, r(u) = 4, |u−u| ≤ 5◦ , u ∈U1, r(u) = 4, 5◦ < |u−u|1.1, r(u) = 5, |u−u| ≤ 5◦

1, r(u) = 5, 5◦ < |u−u|A justification for the selection of weights as above is not given in (Archibald et al.,

2008). The map σS is normalized to get a probability distribution like structure.

Step 5: The full selectability is computed as follows. The UAVs in priority list Pi of

Ui are partitioned into 5 sets Sik,k = 1, . . . ,5 based on the heading choice for which an

aircraft in the list has highest base selectability. For example, if Uj ∈Pi has uk as the

heading choice with highest base selectability, then Uj ∈ Sik. We have

Pi =⋃k

Sik

For each Ui a weight matrix

Wik(u,v) = ∑Uj∈Si

k

WS(Ui,Uj) , u,v ∈U

is calculated where, WS(Ui,Uj) = 1 if there is no conflict (dmin(i, j) > Rnm) when Ui

follows u and Uj follows v, and 0 otherwise. Wik is normalized over each column. Let

W ig(k) denote the cardinality of Si

k. Conditional selectability of Ui is

ρSi(u|u1,u2,u3,u4,u5) ∝5

∑k=1

W ig(k)Wik(u,uk)

where u1,u2,u3,u4,u5 ∈U . For the sake of brevity define U = U ×U ×U ×U ×U

and u = {u1,u2,u3,u4,u5}. The marginal selectability is

ρSi(u) = ∑u∈U

ρSi(u|u)σS1(u1) . . .σS5(u

5)

We use the following values

σS1 = [0.3333 0.2222 0.2222 0.1111 0.1111]

σS2 = [0.2198 0.3297 0.2198 0.1209 0.1099]

σS3 = [0.1099 0.2198 0.3297 0.2198 0.1209]

σS4 = [0.1111 0.1111 0.2222 0.3333 0.2222]

σS5 = [0.1235 0.1235 0.1358 0.2469 0.3704]

47

which is one of the mass function set constructed by the procedure given above. The

marginal selectability ρS is then normalized.

Step 6: Obtain the full selectability function as

PSi = ησSi +(1−η)ρSi

where a η = 0.001 is chosen as in reference (Archibald et al., 2008).

Step 7: Now the choice of heading direction for Ui is

u∗i = argmaxu∈U

(PSi(u)−PRi(u))

This gives the increment in heading Δψi = u∗i .

Step 8: Update the heading and position as in Step 6 of RCA-2D.

2.5 Simulation Results

The performance of the algorithms developed is tested through extensive simulations.

This section describes the test cases and presents and compares the results obtained

using the algorithms proposed in this thesis and the SGT method. All results that

are reported here are obtained from the simulations based on the implementation of

algorithms as described in the last section.

2.5.1 Simulation results for two dimensional case

We use the same performance metrics that are used to measure the performance of

collision avoidance algorithm in (Hill et al., 2005) and (Archibald et al., 2008). These

are number of near misses and efficiency, and are explained below for completeness.

Near misses: From a safety point of view, the UAVs are required to keep at least a

minimum distance between each other. Any location of two UAVs within this mini-

mum required separation results in a near miss. A good collision avoidance algorithm

will ensure fewer near misses. The value of required separation (Rnm) considered for

simulations in this section is 5 miles.

Efficiency: It is required that the UAVs reach the destination in minimum possible

time. The collision avoidance maneuver should be minimal so that UAVs arrive at

their destinations as soon as possible. If T aj is the actual flight time of the jth UAV and

T ij is its ideal flight time (that is, the time taken to reach the destination if the UAV were

48

not to make any deviation for collision avoidance), then the efficiency for N UAVs is

Efficiency =1N

N

∑j=1

T ij

T aj

(2.17)

An algorithm with higher efficiency is considered to be better than another algorithm

with similar near misses but a lower efficiency. The lower efficiency algorithm does a

more conservative collision avoidance maneuver. Thus, the efficiency metric gives an

insight into the optimality of the algorithm in terms of time taken to arrive at destina-

tion while avoiding collisions.

The papers (Hill et al., 2005) and (Archibald et al., 2008) use another performance

metric – number of collisions. A collision is deemed to have occurred if two UAVs ap-

proach within Rc = 300 feet of each other. We do not consider the number of collisions

as a performance metric as no collisions were observed in any of the simulations.

The following values are used for all the simulations in this chapter. The air-

craft are assumed to have a velocity of with magnitude equal to 500 miles per hour

(733.33 ft/s). A maximum turn rate of 5◦ per second equivalent to a turn radius of

Rmin = 5/π miles is assumed. The time step Δt is chosen as 1 second.

Figure 2.8 shows simulation results for simple cases of collision avoidance using

Algorithm 2.3 to avoid head-on collisions. We observe from the figure that RCA-2D

can handle these conflicts. However, the real test is the one in which the airspace is

congested with high density traffic.

Random flight test case

It is desirable that a test case for collision avoidance be complex and at the same time

be sufficiently scalable to test the high traffic density cases. The test case of random

flights (Hill et al., 2005; Archibald et al., 2008) is considered for simulations. In this

test case, the UAVs fly from random points on an outer circle, the destination being

random points on an inner circle. The values 120 and 100 miles are used as the radii

of outer and inner circles for simulations presented in this chapter. Figure 2.9 gives

a schematic of the random flight test. Each simulation is done for a specified aircraft

density or the number of aircraft in the concerned airspace. When the simulation be-

gins, a new aircraft is introduced at a random point on the outer circle at every 5 s

interval, which is assigned a random point on the inner circle as destination, till the

required aircraft density is achieved. Whenever an aircraft reaches the destination, it is

removed and a new aircraft is added at a randomly selected point on the outer circle to

49

(a) Head-on conflict

(b) Intersection conflict

Figure 2.8: Collision avoidance in simple conflict situations.

50

Figure 2.9: A schematic of the random flight test case.

keep the number of aircraft in the airspace constant. Once the airspace achieves a re-

quired density, information regarding efficiency and number of near misses is collected

for a simulation time of 50 minutes. Since the test case involves random introduction

of UAVs and random assignment of destination points, it is called a ‘random flights’

test case and the simulation for a particular aircraft density is conducted 20 times and

average values are computed. As there is a randomness involved in this test case, it is

expected that, if the simulations are carried out for a reasonably large number of times,

almost all the possible conflict geometries would be encountered.

It is required to choose the right values for the tuning parameters of the algorithm

RCA-2D to achieve a good performance. Toward this, extensive simulations are done

with varying values of tuning parameters Rdes, Rsen and λ , and the performance in

terms of number of near misses and efficiency are observed. Figure 2.10 gives results

of one such study for a traffic density of 20 aircraft and a constant λ = 0.5. These

plots give the average number of near misses and efficiencies for values of sensor

range radii {6,7,10,20,30,40,50,60,70,80}. As seen from Figs. 2.10(a) and 2.10(b),

the choice of Rdes and Rsen is a tradeoff between number of near misses and efficiency;

the higher the efficiency, the larger the number of near misses. After similar studies

for other aircraft densities, the values Rsen = 10 miles, Rdes = 6 miles and λ = 0.5 were

chosen for the tuning parameters. Figure 2.10(b) shows a decrease in efficiency with

an increase in sensor radius. This is contrary to the intuition that more information

should lead to better efficiency. It is to be noted that the reactive collision avoidance

algorithms developed in this chapter are not optimal algorithms that take complete

51

Table 2.1: Comparison of performance of algorithms RCA-2D and SGT: Test case ofrandom flights (averaged over 20 runs)

Number of SGT RCA-2DAircraft Near Misses Efficiency Near Misses Efficiency

20 1.95 99.45 1.35 99.0940 7.05 97.94 3.75 97.6460 17.85 94.38 12.65 96.39

advantage of all the available information, but a reactive one. For a UAV implementing

these algorithms, larger sensor radius implies more neighbors. Therefore, it may so

happen that in trying to avoid conflicts with this larger class of neighbors, a UAV

may result in taking a roundabout path avoiding the whole neighbor set as a group

while there may be a safe path through them. This behavior results in poor efficiency.

Restricting the sensor radius forces a UAV to ignore those neighbors which are far

away and take the risk of exploring a safe path through the neighbors.

Table 2.1 gives the results for low density (20 aircraft), medium density (40 air-

craft), and high density (60 aircraft) traffic. For comparison, the values obtained by

simulating the Satisficing Game Theory (SGT) based algorithm of Hill et al. (2005)

and Archibald et al. (2008) are also given in the table. They give two versions of

their algorithm – full and simplified (Hill et al., 2005; Archibald et al., 2008). For

the purpose of comparison, the simplified SGT algorithm is used as it is computation-

ally much less demanding while its performance is almost the same as that of the full

version (Archibald et al., 2008). The simplified satisficing approach to aircraft conflict

resolution SGT, and the way it is implemented for the studies in this chapter were given

in Section 2.4.4. Both the algorithms, SGT and RCA-2D, are executed and evaluated

under the same conditions for all test cases. For example, we use the same sequence

of random numbers in setting up the problem, and evaluate the performances using

the same criteria. All the values in Table 2.1 are averages over 20 runs. As seen from

Table 2.1, the number of near misses are less for RCA-2D when compared to SGT,

while the efficiencies are comparable for all the cases. Figure 2.11 gives a snapshot

during one of the simulations with a traffic density of 40 UAVs.

The performance measure ‘efficiency’ is an indicator of the overall cost (sum of

deviations from the nominal paths) of the entire group of UAVs. We do not claim

global optimality of the proposed algorithm. Nonetheless, the high values of efficien-

cies obtained in the simulation results point to a good performance of the proposed

collision avoidance algorithm.

52

0 10 20 30 40 50 60 70 800

2

4

6

8

10

12

14

16

18

20

sensor radius (miles)

near

mis

ses

R

des= 5 miles

Rdes

= 6 miles

Rdes

= 10 miles

Rdes

= 15 miles

(a) Variation of near misses

0 10 20 30 40 50 60 70 8075

80

85

90

95

100

sensor radius (miles)

effic

ienc

y

Rdes

= 5 miles

Rdes

= 6 miles

Rdes

= 10 miles

Rdes

= 15 miles

(b) Variation of efficiency

Figure 2.10: Variation of near misses and efficiency with Rsen and Rdes for a trafficdensity of 20 aircraft.

53

Figure 2.11: A snapshot during the simulation of random flights where UAVs fly fromrandom points on outer to inner circle for a traffic density of 40 aircraft.

For collision avoidance among multiple UAVs, RCA-2D has a great computational

advantage over SGT. Table 2.2 presents the average computation times taken by SGT

and RCA-2D for simulations with various air traffic densities. RCA-2D is about 14–20

times faster than SGT. Simulations were done using MATLAB on Windows platform

operating on a computer with Intel Core 2 CPU at clock speed of 2.4GHz and 2GB of

RAM. There are two parts to the total computational time: (i) the time to update tra-

jectories of the aircraft, which is the same for both the algorithms, and (ii) the decision

time, which is specific to each algorithm. What is reported in Table 2.2 is the average

decision time, that is, the average wall clock time that each program spent in the guid-

ance loop during a run of random flights of 50 minutes simulation time. This gives a

direct comparison between the computational complexities of the two algorithms. The

averages are taken over 20 simulation runs.

Simulations with noisy measurements

To study the robustness of the collision avoidance algorithm, noise is introduced in

the measurement of UAV positions and headings, and its effect on the performance of

the algorithm is studied. In previous simulations, it was assumed that the positions of

neighboring UAVs which are within the sensor range of a particular UAV are exactly

54

Table 2.2: Comparison of computation times of RCA-2D and SGT per simulation:Test case of random flights (averaged over 20 runs)

Number of Computation Time (sec)Aircraft SGT RCA-2D

20 638 3340 1611 10060 2819 206

Table 2.3: Comparison of performance of RCA-2D and SGT: Test case of randomflights with 20 aircraft in the presence of noise in measurement of position of neigh-boring aircraft (averaged over 20 runs)

Std. Dev. of Near Misses EfficiencyNoise (miles) SGT RCA-2D SGT RCA-2D

0 1.95 1.35 99.45 99.090.1 8.65 1.35 99.58 98.990.2 12.50 1.55 99.79 99.080.3 14.10 1.95 99.44 99.02

known to it. This assumption is relaxed with the assumption that the x and y coordi-

nates of other UAVs are measured with a noise which is normally distributed with a

zero mean and a standard deviation of σ (in miles). Table 2.3 gives the performance of

collision avoidance algorithms, both RCA-2D and SGT, for the case of random flights

with an aircraft density of 20 for various values of σ . From the table, it is seen that

while RCA-2D exhibits robustness to noise in position measurement, there is a sudden

degradation in the performance of the SGT in terms of the number of near misses.

The effect of noise in the measurement of heading of neighboring aircraft on the

performance of the collision avoidance algorithm RCA-2D is also studied. During

simulation, normally distributed noise of zero mean and various standard deviations is

introduced to the actual heading of the neighboring aircraft to make the measurement

noisy. The collision avoidance algorithms, RCA-2D and SGT, were implemented with

this noisy measurement. Table 2.4 presents performance results obtained where the

standard deviations are in degrees. Even in this case it is seen that the degradation in

performance of RCA-2D in terms of number of near misses when compared to SGT is

very low.

Scalability Studies

We now carry out the scalability studies for RCA-2D. The test case of random flights

55

Table 2.4: Comparison of performance of RCA-2D and SGT: Test case of randomflights with 20 aircraft in presence of noise in measurement of heading of neighboringaircraft (averaged over 20 runs)

Std. Dev. of Near Misses EfficiencyNoise (degrees) SGT RCA-2D SGT RCA-2D

0 1.95 1.35 99.45 99.095 10.40 1.80 99.92 99.0110 13.30 1.85 99.96 99.0815 14.60 2.25 99.99 99.06

for higher aircraft densities are conducted to analyze how the algorithm scales in terms

of near misses, efficiency and computational time. For the test case of random flights,

aircraft density as high as 200 UAVs per simulation is considered. Figure 2.12 gives

plots of the number of near misses and efficiency versus number of UAVs (aircraft

density) averaged over 20 runs.

Figure 2.12(a) shows how the number of near misses changes as the number of

UAVs in a simulation of the test case of random flights increases. The algorithm scales

almost linearly till around 80 aircraft for which the number of near misses is only 22.

For higher number of UAVs the near misses increases more rapidly. However, the

increase is not exponential. The degradation of efficiency with increase in number of

UAVs is graceful as seen in Fig. 2.12(b).

Figure 2.13 shows how the computational time taken per simulation scales with the

UAV density. Here again, by computational time, we mean the decision time, that is,

the wall clock time per simulation excluding the trajectory update time. From the log-

log plot in Fig. 2.13(b) it appears that the computational complexity of the algorithm is

bounded by O(n1.8) where n is the number of UAVs (aircraft density) in a random flight

test case. The computational complexity of RCA-2D as calculated above, through

simulations, is valid only for the random flight test case for which it is computed.

However, in the worst case, every UAV will have to account for and calculate predicted

near misses with every other UAVs in flight, which implies a computational complexity

of O(n2). Thus, the computational complexity obtained through simulations for the

random flight test case suggests that the random flight test case is close to a worst

case instance for RCA-2D. Figure 2.14 shows the average decision time per UAV per

second (in milliseconds) as a function of the UAV density. Even for an aircraft density

of 200, in the random flight test case, the average decision time per UAV per second is

just about 3.5 milliseconds. This vindicates the scalability of the proposed algorithm.

The linear growth of the curve in Fig. 2.14 indicates that after some high value of

56

20 40 60 80 100 120 140 160 180 2000

50

100

150

200

250

Number of UAVs

Nea

r M

isse

s

(a) Scalability of near misses

20 40 60 80 100 120 140 160 180 20082

84

86

88

90

92

94

96

98

100

Number of UAVs

Effi

cien

cy

(b) Scalability of efficiency

Figure 2.12: Scalability of number of near misses and efficiency with UAV density(averaged over 20 runs).

57

20 40 60 80 100 120 140 160 180 2000

500

1000

1500

2000

2500

Number of UAVs

Ave

rage

dec

isio

n tim

e pe

r si

mul

atio

n (s

ec)

(a) Computational complexity

101

102

101

102

103

104

Number of UAVs

Ave

rage

dec

isio

n tim

e pe

r si

mul

atio

n (s

ec)

simulation dataline with slope = 1.8

(b) Computational complexity: Log-log plot

Figure 2.13: Average decision time per simulation for different UAV densities (aver-aged over 20 runs).

58

20 40 60 80 100 120 140 160 180 2000.5

1

1.5

2

2.5

3

3.5

Number of UAVs (traffic density)

Ave

rage

dec

isio

n tim

e pe

r U

AV

per

sec

ond

(mse

c)

Figure 2.14: Average decision time per UAV per second with increase in traffic density(averaged over 20 runs).

UAV density, the computations cannot be done in real-time (computational time per

UAV per second will become more that a second). However, that aircraft density at

which the computations cease to be real-time is very high, as can be seen from the

extrapolation of the curve in Fig. 2.14, that a UAV mission at that density will anyway

not be physically feasible.

2.5.2 Simulation results for 3D case

In this subsection, we perform simulations to evaluate the performance of the proposed

3D collision avoidance algorithms. The performance indicators are number of near

misses and efficiency, as before. In the 3D case, the safety zone for each UAV is

considered as a ball of radius equal to the required separation (5 miles) centered at a

UAV. After a sensitivity study, similar to that performed for the 2D case, the values of

the tuning parameters were chosen as Rsen = 10 miles, Rdes = 6 miles and λ = 0.5. We

present the test case of random flights as well as cases where the conflicts are genuinely

3D to begin with.

Random flight test case

Simulations are done for the random flights test case involving 20 to 60 aircraft, in

59

−100

−50

0

50

100

−100−50050100

−20

0

20

x−range (miles)

heig

ht (

mile

s)

y−range (miles)

Figure 2.15: A screenshot of UAVs making 3D collision avoidance maneuvers duringrandom flight test case.

steps of 20. Since the flights are random, for each aircraft density, 20 simulations

are done and the average values of number of near misses and efficiency are cal-

culated. The tuning parameters used for all simulations in 3D are Rsen = 10 miles,

Rdes = 6 miles and λ = 0.5, which are the same as those used for the 2D algorithm. A

screenshot during one of the random flight tests is given in Fig. 2.15.

The performance of RCA-3D-O, which pulls a UAV out of the collision plane,

is compared to that of the 2D algorithm developed in Section 2.2 (RCA-2D) for the

test case of random flights. Table 2.5 presents the simulation results. It is seen from

the table that the 3D algorithm performs much better than the 2D algorithm in terms

of near misses; also, the efficiencies obtained are higher. Note that RCA-3D-I is a

3D extension of the 2D algorithm; for planar engagements, it is the same as RCA-

2D. Also, note that the computational effort required for the 3D algorithm is same as

that of 2D algorithm as seen from Table 2.6. The values reported in the table are the

average decision times, that is, the average times spent in guidance loops alone during

simulations.

Test case of perpendicular flow

There are certain test cases where the use of the third dimension naturally leads to

60

Table 2.5: Comparison of performance of algorithms RCA-2D and RCA-3D-O: Testcase of random flights (averaged over 20 runs)

Number of Near Misses EfficiencyAircraft RCA-2D RCA-3D-O RCA-2D RCA-3D-O

20 1.35 0.3 99.09 99.3540 3.75 1.0 97.64 98.0660 12.65 2.6 96.39 96.82

Table 2.6: Comparison of computation time of algorithms RCA-2D and RCA-3D-O:Test case of Random Flights (averaged over 20 runs)

Number of Computation Time (s)Aircraft RCA-2D RCA-3D-O

20 33 3240 100 10360 206 226

much more effective collision avoidance among UAVs. As an example of this, the test

case of perpendicular flow of aircraft is solved using the RCA-3D-O algorithm. This

test case, in which two streams of aircraft approach perpendicular (or at an angle) to

each other, is very challenging to solve in 2D. Mao et al. (2000) addressed this problem

in the literature by using instantaneous heading and velocity changes. Figure 2.16

shows the RCA-3D-O algorithm taking care of such a situation by letting one stream

go above the other.

Test case of perturbed 2D random flights

This test case is a minor modification of the 2D random flight case. Here, the aircraft

are introduced not on the same plane but randomly at different altitudes within a band

of 5 miles altitude. This will ensure that the conflicts that would have occurred in

the planar case would still occur (as the vertical separation is less than the required

separation (5 miles) for near miss) while the engagements are three dimensional.

Table 2.7 summarizes the performance of the 3D algorithms – RCA-3D-I and

RCA-3D-O – for the test case of perturbed 2D random flights. Both the algorithms

perform equally well. Given that RCA-3D-I is the natural extension of RCA-2D, this

result shows that the performance of the algorithm greatly improves when the en-

gagement is not strictly planar, which is a practical case. Although RCA-3D-I works

equally well as RCA-3D-O in this test case where the engagements are nearly planar,

61

−40−20

020

40

−50

0

50−20

−10

0

10

20

heig

ht (

mile

s)

x−range (miles)

y−range (miles)

Figure 2.16: Collision avoidance in 3D using RCA-3D-O – the test case of perpendic-ular flow of aircraft.

Table 2.7: Performance of the 3D algorithms for the test case of perturbed randomflights (80 UAVs, averaged over 10 runs)

Near Misses Efficiency (%)RCA-3D-I RCA-3D-O RCA-3D-I RCA-3D-O

6.3 6.5 95.98 96.31

its performance degrades when the engagements become more three dimensional in

nature. This is presented below.

Test case of modified random flights

In a real world situation, the engagement of UAVs are seldom planar. RCA-3D-I and

RCA-3D-O readily applies also to the cases where engagements are inherently 3D.

To demonstrate this, we modify the test case of random flights to create a situation

in which conflicts are inherently three dimensional. We consider a situation shown in

Fig. 2.17. The UAVs fly between two planes separated by a distance h. The UAVs

appear at random locations on the outer circle of radius rout in one of the planes and

fly toward random points on the inner circle of radius rin on the other plane. The mid-

air conflicts in such a case is purely three dimensional. As in the test case of random

62

rin

rout

h

Figure 2.17: The test case of modified random flights.

Table 2.8: Performance of the 3D algorithms for the test case of modified randomflights (80 UAVs, averaged over 10 runs)

Case Near Misses Efficiency (%)RCA-3D-I RCA-3D-O RCA-3D-I RCA-3D-O

1 2.6 2.4 98.62 98.842 8.3 4.6 97.30 96.963 11.8 6.1 96.17 96.17

flights, a new aircraft is introduced into the airspace every 5 seconds till the required

traffic density is achieved. Thereafter a UAV is replaced by a new one when it reaches

its destination.

We consider three cases for this test case. In Case 1, we consider h = 20 miles,

rin = 100 miles, and rout = 120 miles. Figure 2.6 is actually a snapshot of this test

case with 20 UAVs. In Case 2 we increase the separation between the two planes to

60 miles. To maintain similar traffic density (that is, the number of UAVs per unit

volume) for the same number of UAVs, we reduce rout to 70 miles and rin to 55 miles.

For Case 3 we have h = 100 miles, rin = 40 miles, and rout = 50 miles. Simulations

are carried out for these cases with a traffic density of 80 UAVs each. The test case

of modified random flights is solved using both the RCA-3D-in and the RCA-3D-out

algorithms. Table 2.8 gives the values of near misses and efficiencies averaged over 10

runs for each of these cases.

The average climb/descent rates of UAVs are high in a case where the separation

between the planes is large. As seen from Table 2.8, RCA-3D-O outperforms RCA-

3D-I in such cases where the engagements are highly three dimensional and the 2D

approximation by projecting the velocities on to the collision plane is not that effective.

We have demonstrated through simulations that the algorithm we developed for

63

collision avoidance among multiple UAVs works well with a kinematic model of UAV

with minimum radius of turn constraint. However, for this algorithm to be useful in a

practical situations, we need to show the efficacy of it when implemented on realistic

UAV models. This is dealt with in the next chapter.

2.6 Summary and conclusions

In this chapter, we developed conceptually simple reactive algorithms to achieve col-

lision avoidance among UAVs, flying to their respective destinations, in a multi-UAV

scenario. First, a 2D collision avoidance algorithm was developed. The algorithm

uses concepts from missile guidance theory, like predicted zero effort miss and time-

to-go, to decide the turn rate of a UAV that will lead to deconfliction of a collision

course. This algorithm tries to achieve collision avoidance by increasing the LOS rate

between the UAVs which are on a most imminent collision course. Through simula-

tions, it was shown that this 2D collision avoidance algorithm – RCA-2D – performs

well with lower near misses and higher efficiency in terms of reaching the destination

as quickly as possible while avoiding collisions. The good performance of the algo-

rithm in the presence of noisy measurements was demonstrated through simulations.

The comparison of results with an existing collision avoidance algorithm for multi-

ple UAVs showed that the proposed algorithm performs better and is about 15 times

faster. Next, a 3D collision avoidance algorithm was proposed that takes advantage of

the third dimension and readily applies to cases where the initial engagement itself is

non-planar. Toward this, we introduced the concept of the collision plane and the 3D

collision avoidance maneuver consisted of pulling out from the collision plane. The

3D collision avoidance algorithm – RCA-3D-O – was shown to perform much better

than the 2D algorithm. Another 3D algorithm – RCA-3D-I – which is a natural three

dimensional extension of the 2D algorithm was also proposed where the avoidance

maneuver takes place on the collision plane as opposed to the out of plane maneuvers

of RCA-3D-O. The algorithm RCA-3D-I was shown to perform as well as RCA-3D-O

for three dimensional engagements that are nearly planar.

The proposed algorithms, both 2D and 3D ones, are decentralized as each UAV

implements them with its limited information of its environment gathered through its

sensors. These algorithms are also simple to implement. The main thesis of this chap-

ter is that it suffices, in case of a multiple UAV conflict, for a UAV to avoid the most

imminent near miss to obtain a good collision avoidance performance. This is vindi-

cated by the simulation studies done in this chapter.

64

The collision avoidance algorithms proposed in this chapter were developed based

on the assumptions of ideal kinematics. Therefore, it remains to be shown that these

algorithms perform well when implemented on realistic UAV models. This is dealt

with in the next chapter.

65

66

3 Collision avoidance with realistic UAV models

In Chapter 2, a free-flight airspace occupied by UAVs was considered and algorithmswere proposed which, when implemented by every UAV in the airspace, results inavoiding collisions and reducing near misses among UAVs. In this chapter, these al-gorithms are tested by implementing them on realistic UAV models in a high densitytraffic simulation environment. Proportional-Integral controllers are designed to im-plement the guidance commands given by the collision avoidance algorithms on theUAV models. The results obtained illustrates the efficacy of the proposed algorithmfor collision avoidance in realistic UAVs.

�se of a kinematic UAV model allows for attaining the commanded turn rates

instantaneously. However, this may is not possible in case of a realistic

UAV leading to a possible degradation in collision avoidance performance

of the proposed algorithm implemented on realistic UAV models. In this chapter, we

show that the collision avoidance algorithms that we developed performs well when

implemented on a realistic UAV model. Toward this, we conduct, using a six Degree of

Freedom (DoF) UAV model, several simulations involving various conflict geometries.

This chapter is organized as follows. In Section 3.1 we introduce the 6 DoF UAV

model. Section 3.2 describes the trimming of the UAV model for a steady level flight.

Design of the altitude, attitude, and velocity controllers to implement the proposed 2D

collision avoidance algorithm is discussed in Section 3.3. Section 3.4 gives simulation

results for collision avoidance tests using the 2D algorithm RCA-2D implemented on

realistic 6 DoF UAV models. In Section 3.5, flight path angle, course, and velocity

hold controllers are designed to implement the 3D collision avoidance algorithm RCA-

3D-O on realistic UAV models. Simulation results of the test cases for 3D collision

avoidance algorithm is given in Section 3.6; cases involving heterogeneous and non-

cooperating UAVs are also given in this section. Section 3.7 gives a summary and

concludes the chapter.

3.1 Six DoF equations of motion

We use the standard six degree of freedom aircraft model and follow the notation of

(Stevens & Lewis, 2003). We write the equations of motion with respect to the body

67

X

Y Z

V

a

Figure 3.1: Body axis of an aircraft.

axis (refer Fig. 3.1) of the UAV. We have (u,v,w) as the velocities along the body axis

and (p,q,r) as the angular rates about the body axis. The Euler angles which fix the

orientation of the UAV relative to an inertial frame is denoted by (φ ,θ ,ψ). The angle

of attack is α and side slip is denoted as β . The elevator, aileron, and rudder defections

are (δe,δa,δr) and δT is the throttle setting.

A rigid body in three dimension has three translational and three rotational degrees

of freedom. The three force equations, corresponding to the translational degrees of

freedom, along the body axis of the UAV are

u = rv−qw+1m

Fx

v = pw− ru+1m

Fy (3.1)

w = qu− pv+1m

Fz

where, m is the mass of the UAV and the forces along the X , Y , and Z directions of the

body axis are given as,

Fx = −mgsinθ +12

ρV 2S(CX0 +CXα α +CXq

cqV

+CXδeδe)

+12

ρSprop(k2motorC

2XδT

δ 2T −V 2)

Fy = mgcosθ sinφ +12

ρV 2S(CY0 +CYβ β +CYp

bp2V

+CYr

br2V

+CYδaδa +CYδr

δr)

Fz = mgcosθ cosφ +12

ρV 2S(CZ0 +CZα α +CZq

cqV

+CZδeδe) (3.2)

68

where, the stability derivatives have the usual definitions (Stevens & Lewis, 2003),

an electric motor-propeller combination is assumed to provide thrust, and for a given

angle of attack α , the force derivatives along X and Z directions of body axis can be

found from the lift and the drag derivatives as,

CX0 = −CD0 cosα +CL0 sinα

CXα = −CDα cosα +CLα sinα

CXq = −CDq cosα +CLq sinα

CXδe= −CDδe

cosα +CLδesinα

CZ0 = −CD0 sinα−CL0 cosα (3.3)

CZα = −CDα sinα−CLα cosα

CZq = −CDq sinα−CLq cosα

CZδe= −CDδe

sinα−CLδecosα

Corresponding to the three rotational degrees of freedom of the aircraft, we have the

three moment equations as,

p = Γ1 pq−Γ2qr +Γ3L+Γ4M

q = Γ5 pr−Γ4(p2− r2)+Γ6M (3.4)

r = Γ6 pq−Γ1qr +Γ4L+Γ7M

where, the moments about X , Y , and Z are

L =12

ρV 2Sb2(Cl0 +Clβ β +Clp

bp2V

+Clrbr2V

+Clδaδa +Clδr

δr)

M =12

ρV 2Sc(Cm0 +Cmα α +Cmq

cqV

+Cmδeδe) (3.5)

N =12

ρV 2Sb2(Cn0 +Cnβ β +Cnp

bp2V

+Cnr

br2V

+Cnδaδa +Cnδr

δr)

The coefficients in the moment equations which depend on the moment of inertias of

the aircraft are give as follows.

Γ1 =Jxz(Jx− Jy + Jz)

Γ

Γ2 =Jz(Jz− Jy)+ J2

xz

Γ

Γ3 =Jz

Γ

Γ4 =Jxz

Γ

69

Γ5 =Jz− Jx

Jy

Γ6 =1Jy

Γ7 =(Jx− Jy)Jx + J2

xz

Γ(3.6)

Γ8 =Jx

Γ

where,

Γ = JxJz− J2xz (3.7)

and Jx, Jy, and Jz are the moment of inertias of the aircraft about X , Y , and Z axes and

Jxz is its only non-zero product of inertia.

We have the following three kinematic equations which relates the body rates to

the Euler angle rates.

φ = p+qsinφ tanθ + r cosφ tanθ

θ = qcosφ − r sinφ (3.8)

ψ =qsinφ + r cosφ

cosθ

The force equations (Eq. (3.1)), the moment equations (Eq. (3.4)), and the kinematic

equations (Eq. (3.8)) together form a system of nine coupled nonlinear first order or-

dinary differential equations which governs the aircraft dynamics. The position of the

aircraft can be obtained by appropriately integrating its velocity and is determined by

the navigation equations

x = cosθ cosψu+(sinφ sinθ cosψ− cosφ sinψ)v

+(cosφ sinθ cosψ + sinφ sinψ)w

y = cosθ sinψu+(sinφ sinθ cosψ + cosφ cosψ)v (3.9)

+(cosφ sinθ sinψ− sinφ cosψ)w

h = sinθu− sinφ cosθv− cosφ cosθw

For the simulations done in this chapter, we use the UAV model used in Aviones flight

simulator (MAGICC Lab, BYU, 2010). This is a UAV of span 1.4224m. Weighing

1.56kg, it is a flying wing configuration similar to Zagi – a popular remote controlled

hobby aircraft (Zagi, 2010). The geometric and inertial parameters of this UAV are

given in Table 3.1. Table 3.2 gives the stability and control derivatives of this UAV

model. The control derivatives corresponding to rudder deflection are not given in

70

Table 3.1: The geometric and inertial parameters of the UAV model used in simulations

Parameter Valuem 1.56 kg

J

⎛⎝ 0.1147 0 0.0015

0 0.0576 00.0015 0 0.1712

⎞⎠ kg m2

S 0.2589 m2

b 1.4224 mc 0.3302 m

Sprop 0.0314 m2

ρ 1.2682 kg/m3

CXδT1

kmotor 20

the table as we do not use rudder deflection to control the UAV. The control surface

deflection limits for elevator and aileron are ±40 degrees.

A small UAV, with a span of about 1.5 m, is chosen as the platform to demonstrate

the efficacy of the proposed collision avoidance algorithm as the missions involving

small UAVs in high density traffic environments are more realistic than the high density

traffic missions of large unmanned aircraft.

We apply our collision avoidance algorithms on 6 DoF UAV models. Toward

this, controllers need to be designed that will track the guidance commands issued

by the collision avoidance algorithms. This controller has to be designed to control

the aircraft about the equilibrium point, that is, about a steady level flight. Below, we

give the procedure to find the control surface deflections, throttle settings, and angle

of attack of the aircraft required (trim conditions) to maintain a steady level flight at a

given altitude and velocity.

3.2 Trimming aircraft for a level flight

The nonlinear UAV dynamics, consisting of force (Eq. (3.1)), moment (Eq. (3.4)), and

kinematic equations (Eq. (3.8)), can be compactly written as

x = f(x,δ ) (3.10)

where, state x = [u v w p q r φ θ ψ ] and control δ = [δe δa δT ]. The

variables have their usual meaning.

71

Table 3.2: Stability and control derivatives of the UAV model used in simulations

Longitudinal Value Lateral ValueCoefficients Coefficients

CL0 0.28 CY0 0CD0 0.03 Cl0 0Cm0 0 Cn0 0CLα 3.45 CYβ −0.98CDα 0.30 Clβ −0.12Cmα −0.38 Cnβ 0.25CLq 0 CYp 0CDq 0 Clp −0.26Cmq −3.6 Cnp 0.022CLδe

−0.36 CYr 0CDδe

0 Clr 0.14Cmδe

0.5 Cnr −0.35CYδa

0Clδa

0.08Cnδa

0.06

To trim a UAV, that is, to find the control surface deflections, the thrust throttle

setting, and the angle of attack for a level flight at a specified speed V , we solve the

following optimization problem

min(α,δ ) ‖ x ‖2 (3.11)

subject to x = f(x,δ )

p = q = r = 0

φ = 0, θ = α , ψ = 0

u = V cosα , β = 0, w = V sinα

The constraints in Eq. (3.11) naturally arises from the steady level flight condition. The

above minimization problem has to search in a four dimensional space (α ,δe,δa,δr)

for solutions. This multidimensional optimization problem can be reduced to a one

dimensional search problem as follows.

For an aircraft that is symmetric about the XZ plane, a level flight would mean a

zero aileron deflection, that is, δa = 0. An explicit expression for elevator deflection

in terms of α can be obtained from the q equation in Eq. (3.4) by setting q = 0 and

imposing other constrains as in Eq. (3.11).

δe =−Cm0 +Cmα αCmδe

(3.12)

72

Similarly, an explicit expression for the trim throttle setting can be obtained by setting

the u equation in Eq. (3.1) to zero.

δT =1

kmotor

√2mgsinα−ρV 2S(CX0 +CXα α +CXeδe)+ρV 2Sprop

ρSpropCXδT

(3.13)

Equations (3.12) and (3.13) along with constraints in Eq. (3.11) makes finding the trim

condition a problem of solving the w equation in Eq. (3.1) for α with w = 0. One

way to do it is to solve it as a one dimensional optimization problem by finding that αwhich minimizes w2.

For most of the simulations in this chapter, the nominal flight condition of the

UAVs, which are, in those cases, assumed to be homogenous, is a cruise flight at an

altitude of 1725m and a velocity of 12m/s. Solving the above optimization problem,

we find that in the nominal cruise flight, the UAV is trimmed at an angle of attack of

6.49 degrees (0.1133 radians) with a corresponding elevator defection of 4.93 degrees

(0.0861 radians) and a throttle setting of 0.7423.

3.3 Controller design for 2D collision avoidance

The objective of the controller, in the case of planar collision avoidance, is to hold

altitude and velocity and track the turn rate command issued by the collision avoid-

ance algorithm. We develop three decoupled controllers for holding (1) altitude, (2)

velocity, and (3) attitude (bank angle). Each of these controllers are designed by suc-

cessive loop closure technique. In all these controllers, the outer loops control the

slower variables and the inner loops control the faster ones.

The outermost loop tries to rectify, through feedback, the error between desired

and actual values of the slowest changing variable by computing through proportional-

integral blocks what the desired value of faster variable should be. The inner loops

handles error in faster variables with the innermost loop dealing with the fastest vari-

able.

3.3.1 Altitude hold controller

The altitude hold controller drives to zero any error between the actual and desired

altitudes. The altitude hold controller has the structure as shown in Fig. 3.2. The

altitude error, as seen in the figure, is fed through proportional-integral blocks which

outputs a desired pitch angle. Since a nonzero pitch angle, θref = α , is required to keep

73

KPh

KIh

s

KPq

KIq

s

KPq

KIq

s

UAVModel

href

h-

++

+

+ - -

+++ +

+

qqref

q

de

deref+

- +

Figure 3.2: Altitude hold controller.

KPV

KIV

s

UAVModel

-

+ ++

V

Vref dT

dTref+

Figure 3.3: Velocity hold controller.

a level flight, it is accounted for when computing the error in pitch. The innermost loop

in the altitude controller is the one with the fastest variable q. The trim elevator setting,

δeref is accounted for in the controller (refer Fig. 3.2).

The values of the Proportional-Integral controller gains, obtained after a manual

tuning, are the following: KPh = 5, KIh = 0.5, KPθ = 5, KIθ = 0.5, KPq = 1, and KIq =

0.1.

3.3.2 Velocity hold controller

Figure 3.3 shows the velocity hold controller. As the controller generates commands

about the equilibrium point of steady level flight, the trim throttle setting, δTref , corre-

sponding to reference velocity, Vref = V , is accounted for. The controller gain values

used for simulations are KPV = 5 and KIV = 0.5.

In reality, it is the throttle command which causes an aircraft to climb/descent

while the elevator deflection will cause a change in its pitch attitude. Therefore, ideally,

it is the throttle control that is to be used for altitude hold. However, we use the elevator

deflection as the control in the altitude hold loop and throttle as the control in the

velocity hold loop. The combined effect of the altitude and velocity hold loops will

help the aircraft to keep a steady level flight as described below.

If the aircraft is below the required altitude, then the elevator controller in the

74

KPf

KIf

s

KPp

KIp

s

UAVModel

fref

f-

++

+

-

+ ++

p

da

Figure 3.4: Attitude hold controller.

altitude hold loop pitches the aircraft up. This causes the angle of attack of aircraft,

and thus the drag, to increase. The increase in drag will cause a velocity drop, to

compensate for which the velocity hold loop will increase the throttle, causing the

aircraft to climb the required altitude.

3.3.3 Attitude hold controller

We use bank to turn the aircraft. Thus a required turn rate is obtained by rolling the

aircraft to the corresponding bank angle. Therefore, we design a controller to hold

the commanded bank angle. The controller to hold a commanded bank angle has a

structure shown in Fig. 3.4. It has two loops, of which the outer loop corrects the error

in the bank angle (slow variable) which when fed through the proportional-integral

blocks will demand the required value of the roll rate (fast variable).

The gain values for attitude hold controller that were chosen, by tuning the inner

loop first and then the outer loop by successive loop closure, are: KPφ = 5, KIφ = 0.5,

KPp = 2, and KIp = 0.05.

When a UAV suddenly banks, the lift vector tilts causing the UAV to lose altitude.

The reference altitude is regained by the combined action of altitude and velocity hold

controllers as described earlier.

3.3.4 Testing the response of the UAV augmented with controller

Both the collision avoidance and the Dubins path to destination algorithms, which

are parts of the 2D collision avoidance algorithms, demand specific turn rates. A

steady level turn rate is achieved by maintaining a bank angle while holding velocity

and altitude. Therefore, we test the response of the UAV, augmented with controllers

designed as described above, to bank angle command. We test the UAV response

75

0 5 10−20

0

20

tφ

(deg

)

0 5 101720

1725

1730

t

h (m

)

0 5 1010

15

t

V (

m/s

)

0 5 100

0.5

1

tδ T

0 5 10−40

−20

0

20

40

t

δ e (de

g)

0 5 10−40

−20

0

20

40

t

δ a (de

g)

Figure 3.5: Response (solid line) of 6 DoF UAV model to bank angle command(dashed line).

with a 3-2-1-1 bank angle command. The 3-2-1-1 is a non-periodic rectangular pulse

input which is commonly used during flight tests to excite all the modes of an aircraft.

The UAV’s response to a 3-2-1-1 bank angle command is shown in Fig. 3.5. The 3-

2-1-1 command requires the bank angle to be 15 degrees for the first three seconds,

−15 degrees for the next two seconds, again 15 degrees for a second and −15 degrees

for the subsequent second, and a zero bank angle thereafter. From Fig. 3.5 we observe

that the UAV exhibits a good response to bank angle command although the elevator

deflections get saturated at times. Also, the rate of aileron deflection demanded is

very high. It is to be noted that the velocity and altitude is maintained throughout via

appropriate elevator deflection and throttle commands.

The plot of steady state turn rate, ψ , achieved for different bank angles is given in

Fig. 3.6(a). It is because the steady state turn rate and bank angle have a linear rela-

tionship, as seen from the figure, that we give a bank angle command to obtain a turn

rate of ψ . For a desired turn rate, the corresponding bank angle required can be found

from the plot in Fig. 3.6(a). From the figure, we find the slope of the line as 0.75.

Therefore, we get the achieved turn rate for a bank angle φ as,

ψachieved = 0.75φ (3.14)

The transient of the achieved turn rate for a commanded bank angle of 10◦ is shown

76

in Fig. 3.6(b). The transient response has got a settling time of around 2 s. It was this

transient response that was used to tune the attitude hold PI controller.

3.4 Simulation results for planar collision avoidance

In this section, we test the collision avoidance algorithm developed in Section 2.2 –

RCA-2D, implemented on 6 DoF UAV models with the controller designed in Sec-

tion 3.3, in a multiple UAV scenario. The 6 DoF UAV equations along with the

controller equations are solved simultaneously using a fourth order Runge-Kutta in-

tegration scheme with a time step of 0.01 seconds.

3.4.1 Tailored test cases

Here we present some test cases to show how UAVs avoid multiple collisions in a

multiple UAV free flight scenario. In these test cases, the initial positions and orienta-

tions of the UAVs are selected so that during flight they encounter multiple conflicts.

The UAVs try to maintain a constant speed of 12 m/s and an altitude of 1725 m.

Figure 3.7 shows the trajectories of 3 UAVs in a free flight airspace which resolved

their conflicts using Algorithm 2.3. Shown in Fig. 3.8 are the important states (bank

angle, altitude and velocity) and the corresponding control inputs (aileron deflection,

elevator deflection and throttle setting) for one of the UAVs. From the figure, we

observe that the altitude and velocity were maintained throughout the simulation time.

Also, the bank angle response, as seen in the figure, indicates the collision avoidance

maneuvers that the UAV made.

Similar cases of multiple collision avoidance involving 4 and 5 UAVs are given in

Figs. 3.9 and 3.10, respectively. Table 3.3 gives the initial positions and orientations of

the UAVs involved in the tailored test case involving 5 UAVs. The destination of each

UAV is 200 meters ahead of the initial position in the direction of its initial orientation.

These cases are tailored in such a manner that a UAV avoiding one conflict gets

into another. However, as seen from the figures, all the conflicts are eventually resolved

without any occurrence of near misses. For the simulations in this section, we do not

use an actuator model. A first order actuator model is considered during the controller

design for the implementation of the 3D collision avoidance which is dealt with in

Section 3.5.

77

0 5 10 15 200

5

10

15

φ (deg)

d/dt

(ψ)

(deg

/s)

(a) Turn rate achieved for various bank angles

0 1 2 3 4 5−5

0

5

10

15

20

t (s)

d/dt

(ψ)

(deg

/s)

(b) Turn rate transient for bank angle of 10◦

Figure 3.6: Turn rates (steady state and transient) for 6 DoF UAV model.

78

meters

met

ers

−100 −80 −60 −40 −20 0 20 40

−40

−20

0

20

40

60

Figure 3.7: Collision avoidance of 3 realistic UAVs.

0 2 4 6 8 10−50

0

50

φ

0 2 4 6 8 101700

1710

1720

1730

1740

h

0 2 4 6 8 100

5

10

15

20

t

V

0 2 4 6 8 10−40

−20

0

20

40

δ a

0 2 4 6 8 10−40

−20

0

20

40

δ e

0 2 4 6 8 100

0.5

1

t

δ T

Figure 3.8: Control inputs and corresponding states for one of the UAVs in Fig. 3.7.

79

meters

met

ers

−100 −80 −60 −40 −20 0 20 40 60−80

−60

−40

−20

0

20

40


−80 −60 −40 −20 0 20 40 60 80

−60

−40

−20

0

20

40

60

meters

met

ers

U1

U2

U3

U4

U5


80

Table 3.3: Initial positions and orientations of UAVs in Fig. 3.10

UAV Initial position Initial orientation(x,y) in meters in degres

1 (60,-40) 752 (-30,-70) 903 (-90,-40) 154 (20,70) 2455 (90,10) 195

Table 3.4: Random flight test case for 6 DoF UAV models with inner and outer radiias 4 km and 5 km (values averaged over 10 runs)

No. of without collision avoidance with collision avoidanceUAVs Near misses Efficiency Near misses Efficiency

20 2.3 100 0 99.99940 8.1 100 0 99.99460 22.4 100 0 99.97980 36.9 100 0 99.981

3.4.2 Random flight test case

We consider the test case of random flights described in Section 2.5.1. Throughout

the test, all UAVs try to maintain a constant speed of 12m/s with the aid of velocity

hold loop of the controller and the maximum turn rate is limited to 10 degrees per

second. The performance measures are again near misses and efficiency as defined in

Section 2.5.1. The desired separation parameter, which is a tuning parameter, in the

proposed collision avoidance algorithm is chosen as 20m. The UAVs are assumed to

have a sensor range of 200m. The UAVs can detect the position and velocity of other

UAVs within this range. A near miss is deemed to occur if two UAVs come within

10 m of each other.

Simulations were done for 20, 40, 60, and 80 UAVs. For each case 10 simulations

were done. For the random flight test, we used radii of 4 and 5 km for the inner and

outer circles. There was no near miss in any of the simulations in this case. The results

are shown in Table 3.4. From the table, we observe that for the case of 80 UAVs, the

possible occurrences of about 37 near misses per simulation is reduced to zero, without

much loss in efficiency, when the proposed collision avoidance algorithm is used.

As we see from Table 3.4, the efficiencies achieved are high. This could be because

the area of the test field considered is large. Therefore, we consider a case where the

inner and outer radii are selected as 400 and 500 m. The results for this case is given in

81

Table 3.5: Random flight test case for 6 DoF UAV models with inner and outer radiias 400 m and 500 m (values averaged over 10 runs)

No. of without collision avoidance with collision avoidanceUAVs Near misses Efficiency Near misses Efficiency

20 218.1 100 0.1 96.1540 899.1 100 1.6 89.1760 2027.9 100 1.4 89.11

Table 3.5. As seen from the table, not applying any collision avoidance maneuver will

result in over 2000 near misses for an aircraft density of 60. The application of RCA-

2D reduces the average number of near missed to about 2. However, this reduction in

the number of near misses comes at the cost of efficiency. The efficiency reduces to

about 90% which means that some of the UAVs have to take long detours to avoid near

misses. Nonetheless, even this drop in efficiency is acceptable considering the small

size of the free flight area considered for this study and the achieved reduction in the

number of near misses.

3.5 Controller design for 3D collision avoidance

The three dimensional collision avoidance algorithm demands an acceleration in 3D,

but perpendicular to the velocity vector. This demanded acceleration can be split into

longitudinal and lateral accelerations; correspondingly, the pitch and the turn rates,

respectively. The demanded acceleration can be translated into equivalent γ and ψrequirements, where, γ and ψ are the flight path and the course angles. We propose to

achieve the demanded acceleration via controllers to hold flight path angle and course

angle. In addition, a velocity hold controller will be used to maintain a constant veloc-

ity.

3.5.1 Six DoF model

We consider the same model that is described in Section 3.1. In addition, we consider

the following first order actuator model.

δe = kδe(δec−δe)

δa = kδa(δac−δa) (3.15)

δT = kδT(δTc−δT )

82

where, δec , δac , and δTc are the commanded elevator and aileron deflections and throttle

setting, and kδe, kδa

, and kδTare the corresponding time constants. For the simulation

results presented in Section 3.6, we use kδe= kδa

= kδT= 0.1.

3.5.2 Pitch rate guidance and control loops

We design a controller to hold the flight path angle in a way similar to the altitude

hold controller in Section 3.3.1. A desired pitch rate is demanded by the collision

avoidance algorithm. The demanded γ is integrated to obtain a demanded flight path

angle which is achieved via a flight path angle hold controller. This is illustrated in

Fig. 3.11. In the design of controller, we have assumed that γ = θ −α although this

is not strictly valid in the presence of sideslip. Nonetheless, simulations show that this

does not affect much the performance of the controller. The values of the constants of

the PI-controller are chosen as KPγ = 5, KIγ = 0.5, KPq = 1, and KIq = 0.1.

3.5.3 Turn rate guidance and control loops

The attitude hold controller in Section 3.3.3 is modified by adding an extra loop to hold

the course or the heading angle. This course hold controller is used as the inner loop

of the turn rate guidance loop to attain a desired turn rate as specified by the collision

avoidance algorithm. The turn rate guidance and control loops are shown in Fig. 3.12.

The following values are chosen for the PI-controller constants. KPψ = 2.5, KIψ = 0.1,

KPφ = 2.5, and KIφ = 0.1 KPp = 2.5, and KIp = 0.1.

3.5.4 Response of UAV augmented with controller

Now, we study the response of the 6 DoF UAV model augmented with pitch and turn

rate guidance and control loops as described above. The response to pitch rate and turn

rate commands are analyzed separately. Figure 3.13 shows the response of the control

augmented UAV to a doublet pitch rate demand. The doublet command is as follows:

Starting with a level flight condition, a steady pitch rate of 10 degrees is demanded

for 5 s, a pitch rate of −10 degrees for next 5 s, and back to steady level cruise. From

Fig. 3.13 we observe that the pitch rate demands are achieved with short rise and

settling times and a little overshoot. Figure 3.13 also shows the corresponding control

surface deflections. Note the unsteady nature of the elevator and throttle deflections.

This is because, unlike a constant turn rate, the maneuver consisting of constant pitch

rate is not a steady state motion of the aircraft.

83

X

YZ

V

a

6 D

oF U

AV

Mod

el

KP q

KI q s

KP g

KI g s

1 sG

uida

nce

gde

sire

d

.

qg

-

++

+

-

+

+

+

de

UA

V

posi

tion

and

velo

city

posi

tions

and

vel

ociti

esof

nei

ghbo

ring

UA

Vs

de r

ef

+

Figure 3.11: Pitch rate guidance and control loops.

84

X

YZ

V

a

6 D

oF U

AV

Mod

el

KP p

KI p s

1 sj

KP j

KI j s

-+

+G

uida

nce

jde

sire

d

.

p-

+

+

da

KP f

KI f s

-+

++

f

UA

Vpo

sitio

n an

d ve

licity

posi

tions

and

vel

iciti

esof

nei

ghbo

ring

UA

Vs

Figure 3.12: Turn rate guidance and control loops.

85

0 5 10 15

−10

0

10γ (deg/sec)

0 5 10 15

−10

0

10δe (deg)

0 5 10 15

−10

0

10ψ(deg)

0 5 10 15

−10

0

10δa (deg)

0 5 10 158

10

12

14

V (m/s )

t (s ec)0 5 10 15

0

1

2

δT

t (s ec)

Figure 3.13: UAV response to a doublet pitch rate demand.

Figure 3.14 shows the response of the UAV to a doublet turn rate demand. The

figure also shows the corresponding control surface deflections. The turn rate response

is a bit more sluggish than the pitch rate response since it has higher number of control

loops (refer Fig. 3.12). The behavior can be further tuned by proper choice of controller

constants. However, we do not further emphasize on the control design as, for the

purpose of this thesis, it suffices to show that the guidance algorithms developed are

easily implementable on realistic UAV models.

3.6 Simulations results of 3D collision avoidance

In this section, we present various simulation scenarios using the three dimensional

collision avoidance algorithm implemented on realistic 6 DoF UAV models.

For collision avoidance, the guidance algorithm requires velocity of UAVs in the

inertial frame of reference. The 6 DoF model provides the velocity in the body frame

of reference. The velocity components in the inertial frame of reference is obtained

from that of body frame of reference via the following transformation.⎡⎣ u

vw

⎤⎦

inertial

= Tb2i

⎡⎣ u

vw

⎤⎦

body

(3.16)

86

0 5 10 15

−10

0

10ψ (deg/sec)

0 5 10 15

−10

0

10δe (deg)

0 5 10 15

−10

0

10γ (deg)

0 5 10 15

−10

0

10δa (deg)

0 5 10 158

10

12

14

V (m/s )

t (s ec)0 5 10 15

0

1

2

δT

t (s ec)

Figure 3.14: UAV response to a doublet turn rate demand.

where,

Tb2i =

⎡⎣ cosθ cosψ sinφ sinθ cosψ− cosθ sinψ cosφ sinθ cosψ + sinφ sinψ

cosθ sinψ sinφ sinθ sinψ + cosφ cosψ cosφ sinθ sinψ− sinφ cosψ−sinθ sinφ cosθ cosφ cosθ

⎤⎦

(3.17)

3.6.1 Random flights

We consider the case of random flights as described in Section 3.4.2. UAVs fly from a

random point on the outer circle of radius 500 m to a point on the inner circle of radius

400 m. A specified aircraft density is maintained by replacing the UAVs that reach

the destination by new ones. The near misses and efficiency achieved for various

aircraft density – number of aircraft in the concerned airspace – is given in Table 3.6.

Note that the collision avoidance performance of the 3D collision avoidance algorithm

implemented on realistic UAV models in terms of near misses is same as that of the

2D algorithm (refer Table 3.5). However, the efficiency of the 3D algorithm is much

higher – close to 100%. As the test case of random flights is a planar test case, use of

a 3D algorithm that takes advantage of the otherwise unused third dimension allows

to negotiate conflicts, in a high density traffic, very efficiently. As the UAVs, trying

87

Table 3.6: Random flight test case with 3D collision avoidance for 6 DoF UAV models,and inner and outer radii of 400 and 500 m (values averaged over 10 runs)

No. of UAVs Near misses Efficiency20 0.4 99.9240 1.2 99.8660 2.7 99.79

Table 3.7: Performance of the 3D algorithms in 6 DoF UAV models for the test caseof modified random flights (40 UAVs, averaged over 10 runs)

Case Near Misses Efficiency (%)1 0.7 99.752 0.7 99.64

to avoid each other, using the 3D collision avoidance algorithm, climb to different

altitudes will reduce further conflicts.

3.6.2 Test case of modified random flights

As stated in Chapter 2, the real power of 3D collision avoidance algorithm is in han-

dling inherently three dimensional conflicts, and such a test case – modified random

flights – was presented in Section 2.5.2. We consider two cases. Case 1: h = 10 m,

rin = 300 m and rout = 400 m. Case 2: h = 20 m, rin = 200 m and rout = 300 m. Re-

fer Fig. 2.17 and Section 2.5.2 for clarifications. The average results of Monte Carlo

simulations of the modified random flights with an aircraft density of 40 is given in

Table 3.7.

3.6.3 Perpendicular flow

The test case of perpendicular flow as considered in Section 2.5.2 is considered now.

As expected, one stream of aircraft goes under and the other climbs over the other.

This is shown in Fig. 3.15. This maneuver is more efficient than the aircraft sticking

to the plane and doing an interleaving motion as given in (Mao et al., 2000).

3.6.4 Non-cooperating UAVs

As the collision avoidance algorithms that we proposed in Chapter 2 are reactive, it

works as effectively even if some of the aircraft in the conflict are non-cooperating.

88

Figure 3.15: Collision avoidance in three dimensional perpendicular flow.

By ‘non-cooperating’ UAV, we mean that the corresponding UAV does not alter its

path even if there is any conflict. To illustrate this, we consider the following test

case – the test case of array flow – as illustrated in Fig. 3.16. A collision avoidance is

achieved even if only one set of UAV performs the collision avoidance maneuver. This

is shown in Fig. 3.17. The case where all the aircraft cooperate is shown in Fig. 3.18.

3.6.5 Heterogeneous UAVs

The collision avoidance algorithms work well in the presence of heterogeneous – dif-

ferent speed – UAVs. We consider the test case of random flights with heterogeneous

UAVs. When a UAV is introduced into the airspace, it is randomly assigned one of the

following four – 10, 12, 15, or 18 meters per second – speeds which it will maintain

through the rest of its flight.

The trim conditions – trim angle of attack, trim elevator deflection (δere f ), and trim

throttle setting (δTre f ) – corresponding to different velocities that we consider are given

in Table 3.8. Note that the motor that drives the propeller is so modeled that the throttle

control δT can take values between 0.5 and 2.0. Results of the Monte Carlo runs of

this test case is presented in Table 3.9.

89

Figure 3.16: The test case of array flow where none of the UAVs cooperate resultingin collisions.

Figure 3.17: The test case of array flow where some of the UAVs cooperate.

90

Figure 3.18: The test case of array flow where all the UAVs cooperate.

Table 3.8: Trim conditions corresponding to various steady level cruise velocities

V αtrim δere f δTre f

m/s degrees degrees10 11.43 8.69 0.6612 6.49 4.93 0.7415 2.39 1.82 0.8718 0.14 0.10 1.01

Table 3.9: Random flight test case with 3D collision avoidance, and inner and outerradii of 400 and 500 m, for heterogeneous 6 DoF UAV models (values averaged over10 runs)

No. of UAVs Near misses Efficiency20 0.6 99.9040 1.5 99.8260 2.3 99.75

91


In this chapter, we implemented the collision avoidance algorithms developed in

Chapter 2 on realistic six degree of freedom UAV models augmented with controllers

designed using successive loop closure. The studies in this chapter showed that the

proposed collision avoidance algorithms are easily implementable in realistic UAVs,

and thus hold promise for applications in multiple UAV missions. Although the col-

lision avoidance algorithms were developed assuming kinematic UAV models, their

performances did not degrade upon use in realistic 6 DoF UAV models. We used tai-

lored test cases to validate the implementation of these algorithms, this is, to ensure

that the conflicts are handled correctly. Through simulations, in a random free-flight

high density traffic environment, we showed that the proposed collision avoidance al-

gorithms, when implemented in realistic UAV models, performs well. Also, we carried

out simulations to demonstrate that the collision avoidance algorithms perform well in

the presence of heterogeneous and non-cooperating UAVs.

Having addressed the collision avoidance of multiple UAVs, we now turn our at-

tention to another important problem in multiple UAV missions – coalition formation.

In the next chapter, we consider the coalition formation of UAVs, that can communi-

cate with each other, for target search and prosecute missions.

92

4 Coalition formation with global communication

Many missions involving multiple UAVs require the UAVs to form coalitions in or-der to successfully complete the mission. This chapter looks at the multiple UAVmission to search and prosecute multiple targets and give computationally efficientcoalition formation algorithms. Once the targets are detected by UAVs, coalitionsare formed by UAVs communicating with each other where it is assumed that thecommunication is global or all-to-all. The coalition formation problem is shown tobe NP-hard. This chapter develops a sub-optimal but polynomial time algorithm,and the solution thus obtained is compared with that obtained using Particle SwarmOptimization. Simulations are carried out to study the effects of variation in numberof UAVs and targets on the coalition formation process. The effect of using hetero-geneous UAVs for the search and prosecute mission is also studied.

�he UAVs are usually small, to facilitate portability. Therefore, it may not be

possible for a single UAV to carry all the resources required to successfully

carry out a mission. This necessitates the formation of coalitions of UAVs

to complete certain missions. In this chapter, we in particular consider a multiple UAV

search and prosecute mission. The UAVs perform a distributed search to find targets.

Once a target is detected, the UAVs communicate with each other to form coalitions

such that the combined resources of the coalition are sufficient to prosecute the target.

The chapter is organized as follows. Section 4.1 gives the problem description and

formulation. The coalition formation algorithms for the search and prosecute mission

and their analysis are given in Section 4.2. Section 4.3 gives the algorithms that the

UAVs in a coalition should employ to rendezvous at the target to inflict maximum

damage. Optimal solution to the problem using Particle Swarm Optimization is given

in Section 4.4. Section 4.5 gives the simulation results. Section 4.6 concludes the

chapter with a summary.

4.1 Problem formulation

We consider a scenario in which a search and prosecute mission is carried out on a

battlefield using N heterogeneous UAVs (also called agents). The UAVs are hetero-

geneous as they can carry different types of resources. The quantity of the resources

that a UAV can carry is limited. Some of these resources are consumable, that is, the

93

resources deplete with use, like fuel and weapons, while others, such as sensors, do

not deplete. Each UAV is identified by its unique token number Ui, i = 1, . . . ,N that is

assigned prior to the start of the mission. We assume that there are M targets whose

positions are unknown.

We assume that a UAV, Ui, can carry n types of resources and its resource capability

vector is denoted by RUi of the form

RUi =(

RUi1 , . . . ,RUi

n

), i = 1, . . . ,N (4.1)

where, RUip , p = 1, . . . ,n represents the number of type-p resources held by UAV Ui.

For example, RUi = (4,2,0,6) implies that the UAV Ui has four type-1 resources

(RUi1 = 4), two type-2 resources (RUi

2 = 2), no type-3 resources (RUi3 = 0), and six type-4

resources (RUi4 = 6).

We assume that the UAVs have limited sensor range and do not have a priori knowl-

edge of the target locations and their resources. To detect the targets, the UAVs have

to carry out a search task. Although there are several search strategies for efficiently

finding targets in an area of interest (Sujit & Ghose, 2009, 2010, 2011), and in partic-

ular, search strategies for a search and prosecute mission (George et al., 2011), in this

thesis, we assume that the UAVs perform a random search. Once a UAV Ui detects a

target Tj, we assume that the UAV that detected it can also determine the type of re-

sources required to prosecute the target. If m different types of resources are required

to prosecute target Tj, then the resource requirement vector is represented as

RTj =(

RTj

1 , . . . ,RTjm

), j = 1, . . . ,M (4.2)

where, RTjq , q = 1, . . . ,m represents the quantity of type-q resources required to prose-

cute the target Tj and m≤ n by assumption. For example, RTj = (3,0,5) indicates that

to prosecute target Tj, the UAVs need three type-1 resources (RTj

1 = 3), zero type-2

resources (RTj

2 = 0), and five type-3 resources (RTj

3 = 5).

We assume that each UAV has the capability to communicate with other UAVs.

Initially, we consider global communication where every UAV can communicate with

every other UAV in the operating zone. This assumption is later relaxed in Chapter 5

where we assume that a UAV can communicate with only those UAVs that are within

its finite communication range.

Once a target Tj is located and identified by a UAV Ui, it assumes the role of a

coalition leader and broadcasts the target resource requirement vector (RTj ). The UAVs

in the search region, that are within the communication range of Ui, possessing at least

94

one of the required resources to prosecute the target Tj will respond to Ui with their cost

and resource capabilities. The coalition leader attempts to form a coalition from among

the UAVs that responded such that the combined resource capabilities of the coalition

is greater than or equal to the target resource requirement. After a feasible coalition is

formed, the UAVs in the coalition rendezvous at the target to inflict maximum damage.

Thus, the cost of a UAV to prosecute a target is based on the earliest time of arrival

(ETA) of the UAV at the target location using Dubins curves.

The objective function that we consider to minimize is the time for prosecution

and, subsequently, the time for mission completion. Other possible objectives can be

to minimize fuel consumption, effort, et cetera (George et al., 2011), which we will

not consider in this thesis.

Assume that the UAV team takes τ time units to accomplish a mission of prose-

cuting all the targets. The mission time τ depends on the time taken by the UAVs to

detect the targets which in turn depends on the performance of the search strategy and

the time taken to form coalitions and to prosecute each detected target. The objective

is to accomplish the mission in minimum time. That is, the global objective is

minτ (4.3)

where the minimization is over all possible coalitions and prosecution sequences. Here

τ is bounded above by the sum of τs and ∑Mj=1 τTj

c , where τs is the search time and τTjc

represents the time taken by the coalition to prosecute the target Tj once it is detected.

The mission time τ is not equal to but bounded by the above sum because some of the

target prosecutions can happen simultaneously. It is to be noted that the search time τs

implicitly depends on the coalition formation strategies. This is because the choice of

coalition will decide the UAVs that are free to carry out the search task. For example,

for a given target, out of two coalitions with same prosecution time, the coalition with

lower size will make more UAVs available for search which may lead to a reduction

in the total search time. We attempt to achieve the global objective in Eq. (4.3) by (a)

minimizing the time taken to prosecute each target, that is,

minτTjc , j = 1, . . . ,M (4.4)

where the minimization is over all possible coalitions, and (b) by minimizing the size

of the coalition formed to prosecute it. Also, a simultaneous strike is desirable to

successfully prosecute the target. Thus, the task of a coalition leader is to form a coali-

tion such that (i) the target is prosecuted in minimum time, (ii) the coalition contains

minimum number of UAVs, and (iii) the coalition members simultaneously prosecute

95

the target. The objective (i) (minimizing the time-to-attack the target) is a desirable

quantity in achieving the objective given in Eq. (4.4), the objective (ii) (which is to

minimize coalition size) ensures that the UAVs distribute their search effort so that the

targets are detected quickly and the mission is accomplished faster, and the objective

(iii) induces maximum damage to the target.

The coalition leader who forms a coalition to prosecute a target is that UAV which

detected the target. The role of the coalition leader Ui that detected target Tj is to form

a coalition C (Ui,Tj), with combined resources RC (Ui,Tj) ≥ RTj , to prosecute Tj taking

into account (i)–(iii). Let the set of all UAVs that responded to the coalition formation

request of Ui to prosecute target Tj, the potential coalition members, be denoted by

PC (Ui,Tj). Let D(Ui,Tj) = {DTjU : U ∈PC (Tj,Ui)} represent the set of earliest time

of arrivals for the UAVs in the set of potential coalition members, and DTjU denotes the

earliest time of arrival (ETA) of the UAV U at the location of target Tj using Dubins

curves (Dubins, 1957). The ETA is used as the cost metric. To determine a coalition,

the coalition leader solves the following optimization problem.

argminC ′(Ui,Tj)⊆PC (Ui,Tj)

(max

U∈C ′(Ui,Tj)D

TjU

)(4.5)

subject to

(∑

U∈C ′(Ui,Tj)RU

p

)≥ R

Tjp , for all p = 1, . . . ,m (4.6)

The solution set C (Ui,Tj), to this optimization problem is the smallest coalition which

satisfies Eq. (4.6), that is, if C ′′(Ui,Tj) ⊆ PC (Ui,Tj) is another set which is a so-

lution to Eqs. (4.5) and (4.6), then |C (Ui,Tj)| ≤ |C ′′(Ui,Tj)|. The set obtained as a

solution to the above optimization problem satisfies the objectives (i) and (ii). It is to

be noted that the optimum value of objective function in Eq. (4.5) is the earliest time

at which all the members of the coalition, C (Ui,Tj), can arrive at the target, Tj, which

is equal to τTjc , the time taken by coalition to prosecute Tj. Therefore, solving the op-

timization problem in Eqs. (4.5) and (4.6) for every target will minimize the objective

function given in Eq. (4.4). Also, the coalition members have to prosecute the target

simultaneously (objective (iii)). In order to prosecute simultaneously, the UAVs have

to arrive at the target location at the same time. The minimum time (objective (i)) for

all the UAV in a coalition to arrive at the target is determined by the UAV that has

the highest ETA to target. As we need to determine both the smallest coalition and

the coalition that can prosecute that target in minimum time, the objective function is

coupled. This coupling makes the optimization problem nontrivial to be solved using

standard techniques. We call this optimization problem the minimum time minimum

member coalition formation problem and show that it is NP-hard.

96

Theorem 4.1 The problem of finding the minimum-time minimum-member coalition

that can prosecute a target with a given resource requirement is NP-hard.

Proof: We show this by reducing, in polynomial time, the NP-hard bottleneck travel-

ing salesman problem to an instance of the minimum time minimum member coalition

formation problem; the decision version of the bottleneck traveling salesman problem

is NP-complete. The bottleneck traveling salesman problem can be stated as follows.

Find the Hamiltonian cycle in a weighted graph which minimizes the

weight of the most weighty edge of the cycle (Garey & Johnson, 1979).

The bottleneck traveling salesman problem is NP-hard (Garey & Johnson, 1979). For

a given graph (V ,E ) with |V | = m and |E | = N, we can assume, without loss of

generality, that V = {1,2, . . . ,m} and E = {1,2, . . . ,N}. Let each vertex denote a type

of resource, and let each edge correspond to a UAV. For an edge k joining vertices i and

j, the corresponding UAV Uk will have one i-type resource and one j-type resource.

Thus the resource vector of Uk is RUk = (0,0, . . . ,0,1,0, . . . ,0,1,0, . . . ,0); with 1’s at

ith and jth positions and 0’s elsewhere. Let the value of the edge k correspond to the

earliest time of arrival of the corresponding UAV Uk. Define a target with resource

requirement of two of each type of resources, that is, the resource requirement is the

m-tuple (2,2, . . . ,2). Given this correspondence, it follows that the minimum time

minimum member coalition that can prosecute the target corresponds to a Hamiltonian

cycle with least value for the most weighted edge.

The target resource requirement of 2 for each resource type along with the fact

that every UAV has at most one resource of any given type will ensure that all the

vertices are visited, that is, every vertex will have at least two associated edges. The

minimum member requirement will ensure that no more than two UAVs with same

type of resource appear in the coalition, that is, no more that two edges are associated

with every vertex. The above two together gives a Hamilton path. The minimum time

requirement of the coalition will ensure that the weight of the most weighty edge of

the corresponding Hamiltonian is the least of all possible Hamiltonian cycles.

Thus the optimization problem that we are tackling is NP-hard and therefore tough

to solve; any exact solution to the problem will not scale well. The solution to the

optimization problem (Eqs. (4.5) and (4.6)) can be found by searching the complete

solution space which is computationally expensive. The computational effort, in that

case, increases exponentially with the number of UAVs and targets, since the problem

97

is NP-hard as shown above, unless P=NP. Since the coalition formation algorithm that

we develop is indented to be used in high density traffic environments, it necessarily

has to be scalable. Therefore, we give a sub-optimal, but polynomial time, algorithm

for the minimum-time minimum-member coalition formation problem. Our solution

handles the problem by decomposing it into two-stages. In the first stage, we will

determine a coalition set that can prosecute the target in minimum time, and using this

set we determine the smallest size coalition that satisfies the constraints. We develop

both a suboptimal polynomial time algorithm as well as an optimal algorithm. In the

next section, we describe the two-stage coalition formation algorithms.

The UAV that detects a target becomes the coalition leader and forms the coalition.

Hence, the coalition formation algorithms are decentralized by nature which has sig-

nificant advantages over centralized solution concepts. However, if the target locations

and their resources, the UAV positions and their resources are known a priori, then we

can solve the global optimization problem off-line. The global optimization problem

has to determine the appropriate coalitions for each target taking the reduction of the

UAV resources into account. The global solution should also determine the order in

which targets are prosecuted. In order to find a solution to this highly complex problem

and to take the dynamics of the resource depletion into account, we propose a Particle

Swarm Optimization (PSO) based solution which we describe in Section 4.4.

The UAVs are subjected to kinematic constraints preventing instantaneous course

changes. We assume that the autopilots of the UAVs hold a constant altitude and

ground speed. The kinematics of the ith UAV is therefore modeled using first order

kinematics as

xi = Vi cosψi,

yi = Vi sinψi,

ψi = k(ψdi −ψi), i = 1, . . . ,N (4.7)

where, xi and yi gives the UAV location, ψi is its heading, ψdi is the desired heading,

Vi is the ground speed, and k is an autopilot gain. We assume the heading rate to be

constrained within bounds. That is

−ωmax ≤ ψi ≤ ωmax, i = 1, . . . ,N (4.8)

The UAVs perform a search task to detect targets. When a target is found, a coalition

with other UAVs is formed to prosecute the target. In the next section, the coalition

formation process is discussed.

98

4.2 Coalition formation

The heterogeneous UAV swarm has to prosecute the targets by forming coalitions.

A coalition is formed based on the target resource requirements for prosecution. A

coalition is a temporal team and the agent/UAV that initiates a coalition formation

request is the coalition leader while the agents which form the rest of the coalition

are called coalition members. The coalition leader may not necessarily be a part of

the coalition because (a) it does not possess the required resources, or (b) the rest of

the coalition can perform the task without its presence, or (c) the coalition leader is

committed to be a part of another coalition. The role of the coalition leader is to form

a coalition and provide the simultaneous strike information to the coalition members.

Once the coalition is formed and the leader broadcasts the coalition information, its

role ceases.

In the search region, many UAVs can detect targets and assume the role of a coali-

tion leader to determine coalitions for the detected targets. When a coalition leader

announces its target information, all the UAVs that have not been assigned to any tar-

get and which have any of the required resources will respond. Other coalition leaders

may also respond to the coalition formation requests. When an agent detects multiple

targets simultaneously, it randomly chooses one of the targets and becomes coalition

leader to form a coalition to prosecute that target.

If a coalition leader is not able to form a feasible coalition (probably because the

UAVs having required resources are already part of another coalition), then the coali-

tion leader re-broadcasts the coalition request as long as the target is within its sensor

range. Once the target is out of its sensor range, it ‘forgets’ the target and that target

will be prosecuted only when it is detected again and a successful coalition is formed

by some UAV.

Since the number of UAVs available to form coalitions are limited, there can be sit-

uations where none of the coalition leaders are able to form coalitions due to deadlocks.

Deadlock is a situation where multiple coalition leaders attempt to form coalitions but

none of them are able to form a coalition. This situation can occur since the total avail-

able UAV resources (sums of all the UAV resources) are not appropriately distributed

between the coalition leaders. A deadlock usually is eventually solved in time as the

UAVs are in motion, however, this situation causes delay in forming the coalitions and

hence the objective of minimizing the coalition prosecution time may not be achieved

efficiently. Note that the deadlock is different from a situation where the total available

99

agent resources are not sufficient to prosecute a target, in which case, the target will

never be prosecuted. Below, we give a mechanism to break deadlocks and show its

efficacy through an example.

To break deadlocks, as described above, that might occur in the course of a search

and prosecute mission, the UAVs use unique token numbers which are assigned to them

prior to the commencement of the search mission. The token mechanism determines

the preference with which agents respond to simultaneous multiple coalition forma-

tion requests. Thus, if a UAV receives simultaneous requests from multiple coalition

leaders, all of which it can contribute to, then it responds to the coalition leader with

the highest token number.

Other deadlock avoiding mechanisms can be used. For example, consider the ETA

of a UAV to target location broadcast by a coalition leader as the deadlock avoidance

mechanism. Assume that a UAV Uk receives requests from various potential coalition

leaders. Then the UAV Uk will determine ETA for each of the targets and select a

coalition leader which had broadcast the target to which Uk has minimum ETA.

Although using ETA is a simple metric, it may not avoid deadlocks consistently.

For example, consider a situation shown in Fig. 4.1, where UAVs U1 and U2 detect

targets T1 and T2, respectively and need to form coalitions. Let the target resources

be RT1 = (2,3,4), and RT2 = (3,1,2), and UAV resources be RU1 = (1,2,1) ,RU2 =

(2,1,0) ,and RU3 = (1,1,2) ,RU4 = (1,1,1). When the coalition formation request is

broadcast by U1 and U2 for T1 and T2, respectively, then the UAVs U1 and U3 will

respond to proposal of U1 as their ETA to T1 is smaller than ETA to T2. The UAVs

U4 and U2 send their bid to U2, because their ETA to T2 is smaller than to T1. In this

case, the coalition resources of target T1 are RC (U1,T1) = (2,3,3), while that of T2 are

RC (U2,T2) = (3,2,1). We can see that RC (U1,T1) and RC (U2,T2) do not meet the RT1 and RT2

constraints and therefore no coalitions are formed causing a deadlock. This situation

may prevail for some more time until the geometry of the UAV locations changes and

one of the coalition leaders finds sufficient coalition resources to form a coalition.

In the above example if we use the token mechanism then the UAVs will respond

to U2 first as U2 has higher token number than U1. The agents U1,U2,U3, and U4 will

respond to the request and a coalition constituting U2 and U3 is formed. After U2

determines its coalition, U1 will broadcast again and U4 will send its information as a

response to the broadcast of U1. However, the cumulative resources of U1 and U4 do

not satisfy the target requirement for T1, hence a coalition is not formed. Using this

scheme, we can see that at least one coalition has been formed to execute the task as

compared to no coalitions. As long as the sum of resources of the members intending

100

U1

U3 U4

U2

T2

T1

Figure 4.1: A situation where multiple agents detect multiple targets.

to be a part of the coalition is greater than or equal to the resources of the target, the

token mechanism guarantees that at least one coalition is formed. For simulations in

this chapter, we use the token mechanism.

Once a UAV detects a target, it has to form a coalition that depends on the current

state of the UAV. Figure 4.2 shows two different states (S1 and S2) and how the UAV

makes a coalition. The sequence of actions carried out during these states are described

below.

Coalition leader has all the required capabilities (S1)

In this case, UAV Ui detects target Tj that requires RTj resources. If RUip ≥ R

Tjp for all

p = 1, . . . ,m, and if Ui is not already a part of another coalition, then Ui would attack

target Tj without requesting a coalition with other UAVs. This is a special case of

coalition formation where only one UAV comprises the coalition. The UAV determines

the route to travel using Dubins curves (Dubins, 1957), then proceeds toward the target

as shown by the state S1 in Fig. 4.2.

Coalition leader has partial resources or no resources (S2)

This state occurs when a UAV Ui detects target Tj that requires RTj resources, but

Ui has insufficient resources. In this case, Ui assumes the role of coalition leader and

broadcasts the information about the target, that is, its location and required resources,

to the other UAVs. The UAVs that have at least one of the types of required resources

will send their cost to arrive at the target (ETA) and their available resource vector. The

coalition leader considers all the responses and determines whether a coalition can be

101

In c

oalit

ion?

Det

ect T

j and

dete

rmin

e RT j

Ui h

as a

ll th

ere

sour

ces

Ui h

as in

suff

icie

ntre

sour

ces

Det

erm

ine

path

to ta

rget

and

pros

ecut

e it

Bro

adca

st T

j and

RT j

Rec

eive

pro

posa

ls

for c

oalit

ions

Coa

litio

n fe

asib

le?

Bro

adca

st

'dis

card

coa

litio

n'

Det

erm

ine

and

broa

dcas

t t cT j

and

C (U

i ,T j

)

Che

ck if

Uk

has a

ny o

f the

re

quire

d re

sour

ces

for T

j

No

coal

ition

inte

rest

Cal

cula

te D

T j a

nd

broa

dcas

t alo

ng w

ith R

Uk

Uk

No

coal

ition

inte

rest

Det

erm

ine

path

to ta

rget

and

pros

ecut

e it

Dec

isio

n pr

oses

s for

coa

litio

n le

ader

Ui

Dec

isio

n pr

oses

s for

pot

entia

l coa

litio

n m

embe

r Uk

To o

ther

UA

Vs

From

oth

er U

AV

s

To o

ther

UA

Vs

S1

S2

Yes

No

Yes

No

Yes

No

Figure 4.2: Sequence of events during coalition formation.

102

formed. If a coalition cannot be formed, then it sends a ‘discard coalition’ broadcast

as shown in Fig. 4.2, otherwise, it will form a coalition and broadcast the coalition

information along with the information about the time at which they are expected to

rendezvous at the target. The selected coalition members will replan their paths using

simultaneous strike mechanism that we develop in Section 4.3) to prosecute the target.

The rejected agents will discard their potential interest in the coalition and continue to

perform their search task. During this process, the potential members who announced

their availability for coalition will be in a wait state to receive information from the

coalition leader. The complete sequence of actions that happens during the coalition

formation for a target by the coalition leader as well as by other UAVs are shown in

Fig. 4.2.

The coalition leader has to solve the optimization problem described by Eqs. (4.5)

and (4.6) to determine the coalition members. Since solving the optimization prob-

lem is computationally intensive, we develop two coalition formation algorithms: (i)

polynomial time coalition formation algorithm (which is sub-optimal) and (ii) opti-

mal coalition formation algorithm. Both the algorithms use a two-stage mechanism to

produce solutions that have low computational complexity.

4.2.1 Polynomial time coalition formation algorithm (PTCFA)

Determining the smallest coalition that would successfully prosecute the target in

minimum time can be accomplished in two stages. In the first stage, we determine the

set of UAVs that can achieve the minimum time requirement and then prune this set

to achieve the minimum member coalition in the second stage. The process to achieve

this task is given in Algorithms 4.1 and 4.2. We assume that UAV Ui is the coalition

leader and it has detected target Tj that requires RTj resources.

Algorithm 4.1 begins by initializing the coalition set to empty and the coalition

resources to zero (line 1). First, the responses of the potential coalition members are

sorted in the ascending order of their ETA to target (line 2). The algorithm takes one

UAV (Uk) at a time (line 4) from the ordered list, include Uk in the coalition C (Ui,Tj)

(line 5), update the coalition resource vector RC (Ui,Tj) (line 6) and the coalition time

τTjc (line 7). Then it checks whether the target resource constraint is met (line 8). If the

constraint is not met, then the process of including the next agent and its resources and

verifying the resource constraint continues until the target resource constraint is met

or the UAVs in the sorted potential coalition member set is exhausted in which case

a feasible coalition is not formed (line 17). Once the resource constraint is met, the

algorithm returns the feasible coalition C (Ui,Tj) and the coalition time τTjc (line 15).

103

Algorithm 4.1: First stage of the PTCFA to form a minimum time coalition

Input: Potential coalition members PC (Ui,Tj) and their ETAs D(Ui,Tj)

Output: Coalition C (Ui,Tj) and coalition time τTjc

1: C (Ui,Tj) = /0 and RC (Ui,Tj) = 0

2: [Dsorted, PCsorted] = sortAscending(D(Ui,Tj),PC (Ui,Tj))

3: for l = 1 to |PC (Ui,Tj)| do

4: Uk←PCsorted(l)

5: C (Ui,Tj)← append Uk

6: RC (Ui,Tj)← RC (Ui,Tj) +RUk

7: τTjc ← Dsorted(l)

8: if RC (Ui,Tj)p ≥ R

Tjp , for all p then

9: break

10: else

11: continue

12: end if

13: end for

14: if RC (Ui,Tj)p ≥ R


15: return C (Ui,Tj) and τTjc

16: else

17: return No feasible coalitions

18: end if

104

Algorithm 4.2: Second stage of the PTCFA to prune members to obtain a reducedmember coalition

Input: Minimum time coalition C (Ui,Tj) from Algorithm 4.1

Output: Pruned coalition C (Ui,Tj)

1: for l = 1 : |C (Ui,Tj)| do

2: Uk← C (Ui,Tj)(l)

3: RC (Ui,Tj)← RC (Ui,Tj)−RUk

4: if RC (Ui,Tj) ≥ R


5: C (Ui,Tj)← C (Ui,Tj)\Uk

6: RC (Ui,Tj)← RC (Ui,Tj)

7: end if

8: end for

9: return C (Ui,Tj)

To illustrate Algorithm 4.1, we give a hypothetical example where UAV U1 detects

a target T1 that requires only two types of resources. The target resource requirement

is RT1 = (5,3), with RT11 = 5 and RT2

2 = 3. Suppose that the UAVs that responded

to the coalition leader with their resources and cost are given in Table 4.1. As per

Algorithm 4.1, the possible coalition members are sorted in ascending order of their

cost. The new list is given in Table 4.2.

The ordered set of coalition members is PC sorted = {U2,U3,U6,U1,U4,U5}. The

coalition is formed by recruiting members from this ordered set starting with the first

member. The progress of the coalition formation process is as follows. Initially U2,

the first member in the ordered set is selected into the coalition making the coalition

C (U1,T1) = {U2} with RC (U1,T1) = (1,3). Since RC (U1,T1) is not sufficient to prosecute

the target, the next UAV, U3, is added to C (U1,T1) forming C (U1,T1) = {U2,U3} with

combined resources RC (U1,T1) = (2,4). As RC (U1,T1) is still not sufficient, the next UAV

U6 is added to C (U1,T1) which becomes {U2,U3,U6} and the resource vector of coali-

tion becomes RC (U1,T1) = (2,6). The coalition resources are still not sufficient, hence

105

Table 4.1: List of the agents which responded to the coalition formation request in thegiven example, their available resources and costs to the target

UAV Resources cost in secondsU1 (2, 1) 123U2 (1, 3) 47U3 (1, 1) 63U4 (2, 0) 172U5 (3, 2) 207U6 (0, 2) 96

Table 4.2: List of the agents in the given example, after sorting

UAV Resources cost in secondsU2 (1, 3) 47U3 (1, 1) 63U6 (0, 2) 96U1 (2, 1) 123U4 (2, 0) 172U5 (3, 2) 207

the next UAV in the list, U1, is included in C (U1,T1) resulting in RC (U1,T1) = (4,7).

The resource RC (U1,T1) is still short of the requirement and U4 is recruited to the coali-

tion resulting in C (U1,T1) = {U2,U3,U6,U1,U4} and RC (U1,T1) = (6,7), which satisfies

the target resource requirement. The Algorithm 4.1 terminates once it determines the

coalition that satisfies the target resource constraint. The prosecution time for simul-

taneous rendezvous is determined by the coalition member whose cost (ETA) is the

highest. In the present case, the ETA of U4 is the highest with 172 s. Algorithm 4.1

will ensure a coalition with minimum time to target. This is proved in the following

theorem.

Theorem 4.2 Algorithm 4.1 generates an optimal minimal time coalition set.

Proof: Let N be the number of agents that responded to a coalition formation request

by Ui to prosecute target Tj that requires resources RTj . Let the set of potential coalition

members be given by PC (Ui,Tj) = {Ur1 , . . . ,UrN}, where r1, . . . ,rN ∈{1, . . . ,N}, with

corresponding costs (ETA to target) as D(Ui,Tj) ={

DTjUr1

, . . . ,DTjUrN

}. Assume that

∑U∈PC (Ui,Tj) RU ≥ RTj . Algorithm 4.1 orders the set PC (Ui,Tj) to form PCsorted =

{Us1 , . . . ,UsN} where s1, . . . ,sN is a permutation of r1, . . . ,rN such that the correspond-

106

ing ordered cost Dsorted ={

DTjUs1

, . . . ,DTjUsN

}has the property that D

TjUsk≤ D

TjUsk+1

,∀k ∈{1, . . . ,N− 1}. For a given l ∈ {1, . . . ,N}, let PC l

sorted = {Us1 , . . . ,Usl} be the first l

elements of the ordered set PCsorted . The target prosecution time for PC lsorted is D

TjUsl

.

The algorithm then finds the least l such that ∑U∈PC lsorted

RU ≥ RTj . Let lmin be the least

integer in {1, . . . ,N} with the above property. Then PC lminsorted is the coalition formed

by Algorithm 4.1 and the claim is that this is a coalition that prosecutes the target Tj

in minimum possible time. Clearly, PC lminsorted is a feasible coalition as the combined

resources of the coalition are enough to prosecute the target. Since the elements of

Dsorted are ordered in ascending order, the addition of an agent Usk with k > lmin will

not decrease the coalition time. Similarly, removal of any agent from PC lminsorted other

than Uslminwill not decrease the coalition time which is D

TjUslmin

. Whereas, removal of

Uslminwill result in an infeasible coalition as lmin is the least integer such that PC lmin

sorted

satisfies the resource requirement. Thus PC lminsorted is a feasible coalition with minimum

time to prosecute target Tj.

Once the minimum time coalition is formed by Algorithm 4.1, we need to prune those

members whose resources are not required to form a coalition with least size. However,

finding a minimum member coalition is NP-hard as shown below.

Theorem 4.3 The problem of finding minimum member coalition is NP-hard.

Proof: We prove this by reducing the set covering problem to an instance of the

minimum member coalition problem. The set covering problem can be defined as

follows.

Find a sub-collection of least cardinality that covers the given collection

of finite sets.

Set covering problem is NP-hard (Cormen et al., 2001). Let S1,S2, . . . ,SN be the given

sets. Let S =⋃N

i=1 Si. The set coveing problem requires us to find a Γ ⊂ {1,2, . . . ,N}of minimum cardinality such that

⋃γ∈Γ Sγ = S.

Let the cardinality of S be m. Without loss of generality, the elements of the set S

can be represented as {1,2, . . . ,m}. Let each element correspond to a type of resource.

Let each set Si, i ∈ {1,2, . . . ,N} correspond to a UAV Ui whose resource capability is

defined as follows. A UAV Ui has one p-type resource, for p = 1,2, . . . ,m, if and only

if p∈ Si. Thus, if RUi = (RUi1 ,RUi

2 , . . . ,RUim ) is the resource capability of Ui, then RUi

p = 1

if p ∈ Si, and 0 otherwise for p = 1,2, . . . ,m. Define a target that requires one resource

of each type, that is, the target resource vector is (1,1, . . . ,1).

107

With this correspondence, the minimum member coalition that can prosecute the

target corresponds to the minimum set cover of S. Since one each of every type of

resource is required, the resultant coalition that can prosecute the target will not only

be of minimum size but also covers the set S.

Since finding a minimum member coalition is NP-hard, it may not be possible to de-

termine this coalition without exhaustively searching all possible coalitions. However,

this will be computationally expensive, especially for the dense traffic situations that

we consider. Therefore, we give a sub-optimal, but polynomial time, algorithm to

obtain a coalition which has lesser number of members than the coalition formed by

Algorithm 4.1. This is given in Algorithm 4.2. The algorithm checks for each UAV Uk

in C (Ui,Tj), the coalition formed by Algorithm 4.1, whether the resources of Uk are

required for the coalition or not by removing its resources from RC (Ui,Tj) (line 3). If

the resources of Uk are not required, then the UAV Uk is removed from C (Ui,Tj) (line

5) and its resources are deducted from the RC (Ui,Tj) (line 6). This process is carried

out for all the UAVs Uk ∈ C (Ui,Tj). The Algorithms 4.1 and 4.2 together forms the

two-stage Polynomial Time Coalition Formation Algorithm (PTCFA).

We continue the example given above to illustrate the second stage of the coali-

tion formation given by Algorithm 4.2. The input to Algorithm 4.2 is the mini-

mum time coalition found by Algorithm 4.1. For the example we consider, this is

C (Ui,Tj) = {U2,U3,U6, U1,U4} with combined resources RC (Ui,Tj) = (6,7). The first

UAV in C (Ui,Tj) is U2. Therefore, its resources are removed from RC (Ui,Tj) to get

RC (Ui,Tj) = (5,4). As the target requirement is RT1 = (5,3), the condition R

C (Ui,Tj) ≥RT1 is met, and hence the UAV U2 is removed for the coalition and its resources

from the coalition resource vector. This results in C (Ui,Tj) = {U3,U6,U1,U4} and

RC (Ui,Tj) = (5,4). The next UAV in C (Ui,Tj) is U3, and with its resources removed

from RC (Ui,Tj), we get RC (Ui,Tj) = (4,3). Since, removing UAV U3 from C (Ui,Tj) re-

sulted in not meeting the target resource constraint, U3 and its resources are restored

in the coalition. Removal of any of the remaining UAVs in C (Ui,Tj) will cause a vio-

lation of the target resource requirement. Hence, we get the coalition after the second

stage as C (Ui,Tj) = {U3,U6,U1,U4} with resources RC (Ui,Tj) = (5,4). Determining a

feasible coalition set with a small size as above involves less computational complexity

but can produce sub-optimal solutions as all possible sub-coalitions and their resource

contributions are not considered. Therefore, below we give another algorithm that will

find an optimal minimum member coalition.

108

Algorithm 4.3: Optimal coalition formation algorithm

Stage 1: Use Algorithm 4.1 to determine C (Ui,Tj)

Stage 2: Solve

argminC ′(Ui,Tj)⊆C (Ui,Tj)

|C ′(Ui,Tj)| (4.9)

subject to

(∑

U∈C ′(Ui,Tj)RU

p

)≥ R

Tjp , for all p = 1, . . . ,m (4.10)

4.2.2 Optimal coalition formation algorithm (OCFA)

To determine the coalition which has an optimal size also, we begin with the minimum

time coalition generated by Algorithm 4.1. Next, we prune the UAVs from this set to

obtain the optimal member coalition. To achieve this, in the second stage, we formulate

an integer programming problem to determine the least number of agents that have

sufficient resources to prosecute the target. The formulation of the problem is given in

Algorithm 4.3.

Algorithm 4.1 provides optimal minimum time and the Stage 2 of Algorithm 4.3

generates an optimal minimum size coalition. Hence the two-stage algorithm (Al-

gorithm 4.3) generates an optimal minimum time and minimum size coalition. We

continue the example given above to describe the functioning of the second stage of

the optimal coalition formation algorithm. The optimal coalition obtained after solving

Eqs. (4.9) and (4.10) as an integer programming problem is C (Ui,Tj) = {U2,U1,U4}with RC (Ui,Tj) = (5,4).

From the coalitions obtained using PTCFA and OCFA, we observe that the coali-

tion formed using OCFA has the smaller size with 3 members and it is the optimal

solution for a single target. The coalition formed using PTCFA is sub-optimal as it

may not always find a minimum member coalition although its solution is always time

optimal.

4.2.3 Complexity analysis

Now we analyze the complexity of the PTCFA and OCFA algorithms. The PTCFA

produces a sub-optimal solution but it has polynomial time complexity. Hence, it

109

can be used for real-time applications. Below, we give a result on the complexity of

PTCFA.

Theorem 4.4 The computational complexity of the PTCFA is O(N(logN +m)).

Proof: Assume that N UAVs respond to the coalition request and the target Tj re-

quires m types of resources to prosecute. The first stage (using Algorithm 4.1), which

includes the sorting of proposals requires O(N logN) computations. The steps 7 and

9 (addition and comparison of m dimensional vector) takes Nm computations each.

Hence the computational complexity of the first stage is O(N logN)+O(Nm). The sec-

ond stage (using Algorithm 4.2) requires Nm computations each for steps 4 and 5 (sub-

straction and comparison of m dimensional vector), that is, a computational complexity

of O(Nm). Therefore, the algorithm complexity of PTCFA is O(N logN)+O(Nm).

The computational complexity for the optimal coalition formation algorithm (OCFA)

has a polynomial time complexity for the first stage as the first stage of OCFA is same

as that of PTCFA. But we use an integer programming technique to solve the optimiza-

tion problem in the second stage of OCFA. Although solving an integer programming

problem is NP-hard, there are pseudo-polynomial algorithms to generate optimal so-

lutions (Papadimitriou & Steiglitz, 1982). To solve the integer programming problem

in the second stage, we used the bintprog command in MATLAB that in turn uses

a branch and bound technique to compute the solution.

When N is very large, the computational time for optimal size coalition depends

on two factors: the number of UAVs and the quantity of their resources. During the

initial phase of the mission, the UAVs have resources in good quantity that the coalition

leader may receive a higher number of proposals. However, since every UAV is full

of resources, the coalition selected after stage 1 will have lower number of UAVs. As

the minimum member coalition is to be found from a subset of this set which is small,

the computational time will be small even if OFCA is used. The case is reversed

during the final stages where the UAVs may have fewer resources and the coalition

size is necessarily larger to meet the target resource requirements. As the coalition size

becomes large the computational time increases. The above analysis indicates that the

computational time for the optimal member coalition formation mainly depends on the

distribution of the UAV resources.

110

4.3 Simultaneous strike

The coalition leader Ui determines the latest arrival time τTjc for the coalition and

provides this information to the accepted coalition members. The coalition members

need to adjust their paths such that their time to arrive at the target is equal to τTjc . There

are several methods that can be used to achieve simultaneous strike by solving the

rendezvous problem (Lin et al., 2003; Tiwari et al., 2004; Lin et al., 2005; Notarstefano

& Bullo, 2006). We develop a different approach where there is no communication

between the coalition members to achieve simultaneous strike. The coalition members

need only the information of τTjc from the coalition leader. The UAVs then find the

radius R, bounded below by the minimum radius of turn, of the Dubins curve to be

followed by the UAV such that the ETA of the UAV is equal to τTjc .

Assume that Ui is the coalition leader for target Tj forming a coalition C (Ui,Tj)

with latest arrival time of τTjc . Each member Uk ∈C (Ui,Tj) has to adjust its path length

by finding a turning radius Rk such that DTjUk

= τTjc . Since, Rk cannot be calculated

using a closed form solution, we calculate Rk iteratively until the condition DTjUk

= τTjc

is satisfied.

The selection of the coalition members in the first stage of PTCFA and OCFA is

based on the ETA of the UAV from the target. The choice of ETA and its computation

are explained below.

Given the UAV position with its heading and the target location, we consider two

possible paths to the target: (i) Dubins shortest path and (ii) roundabout path, as shown

in Fig. 4.3(a). Both paths use a minimum turn radius. Although Dubins path is the

shortest distance to target, we choose the roundabout path length as ETA to target for

the following reason. Consider a situation as shown in Fig. 4.3(b) where UAVs U1

and U2 are part of the coalition to prosecute T1 and need to rendezvous at T1. If both

UAVs take Dubins path to target, since DT1U2

> DT1U1

(refer Fig. 4.3(b)), the UAV U1

increases its turn radius to increase its path length to target. However, if DT1U2

is much

larger that DT1U1

, then it can so happen that the turn radius chosen by U1 is big enough

to encircle the target as shown by the dotted circle in Fig. 4.3(b). In that case, the UAV

U1 may have to choose a Dubins path, similar to that in Fig. 2.5(b), which may have a

path length larger than DT1U2

. Thus there can be a discontinuity in the achievable ETA

when one uses the shortest length Dubins path to target. To circumvent this problem,

we propose that the UAVs use the roundabout path to target and its length as ETA to

target. This path length can be increased continuously, by increasing the turn radius,

111

T1

U1

DT1U1

DT1U1

__

(a) Dubins shortest path and the round-about path to a target

T1

U1

DT1U1

DT1U1

__

U2

DT1U2

DT1U2

__

(b) Justification for the choice of roundabout path length asETA

Figure 4.3: Dubins path and the roundabout path to a target.

to match the coalition time τTjc . In this case, any value of τTj

c ≥ DU1Tj

is guaranteed to

be achieved, as opposed to the case where Dubins path is used.

To calculate the ETA of a UAV using the roundabout path from its current position

to a target, we need to consider two cases as shown in Fig. 4.4. Case (a): The target is

on the right hand side of the UAV and the UAV has to turn counter-clockwise to follow

the roundabout path to the target. Case (b): The target is on the left hand side of the

UAV and the UAV has to make a clockwise turn to reach the target using a roundabout

path.

Let (xi,yi) be the location of the UAV and ψi be its heading. Let (xT ,yT ) be the

target location. Let (xc,yc) be the center of the circle tangent to the UAV’s velocity

and drawn away from the target (in the direction of turn). For case (a), this is given as

xc = xi−Rsinψi

yc = yi +Rcosψi (4.11)

and for case (b), this is

xc = xi +Rsinψi

yc = yi−Rcosψi (4.12)

where R is the radius of the circle along which the UAV makes a turn.

112

Ui

(xi,yi)

(xc,yc)

q1q3

q2

j

(xT,yT)

i

(a)

Ui

(xi,yi)

(xc,yc)

q1

-q3q2

j

(xT,yT)i

(b)

Figure 4.4: Calculation of length of roundabout path to target.

In both the cases, we find angles θ1,θ2 and θ3, shown in Fig. 4.4, as follows.

θ1 = arccos

(√(xT − xc)2 +(yT − yc)2

R

)

θ2 =π2−ψi (4.13)

θ3 = arctan

(yT − yc

xT − xc

)

where, θ3 ∈ (−π,π]. For case (a), the ETA is found as

DTUi

= (R{2π− [θ1− (π− (θ2 +θ3))]}+R tanθ1)/Vi (4.14)

and for case (b), it is

DTUi

= (R{2π− [θ1− (θ2 +θ3)]}+R tanθ1)/Vi (4.15)

where Vi is the ground speed of the UAV.

Note that the roundabout path for each UAV in a coalition, for the simultaneous

prosecution of target, can be found by adjusting the corresponding radius. Given the

coalition time τTc for a target T , a UAV Ui adjusts its path length to target, DT

Ui, to

match the coalition time by choosing an appropriate radius of turn. Since the radius

of turn, R, cannot be explicitly solved from Eqs. (4.14) or (4.15), we find it iteratively.

We start with two guess values of radii of turns – one giving a path length greater than

τTc , and other giving a path length less than τT

c . Then, we use a bisection method to

iteratively find a radius of turn which gives the length of the roundabout path to target

close to the coalition time τTc within the desired accuracy.

113

It is to be noted that although the path length of the roundabout path is greater

than that of Dubins shortest path, the additional time required is bounded above by

2πRmin/V where V is UAV’s velocity and Rmin is the minimum radius of turn.

Once the coalition is formed, the UAVs travel along the computed roundabout path

to target. During this process, the coalition members may detect other targets. In that

case, the detecting member will become a coalition leader for that target and carry out

the coalition generation process. However, this UAV will not participate as a member

of the coalition, since it is already a part of another coalition. From the computational

analysis described in the previous section, the two-stage algorithms have low compu-

tational complexity and require less communication – only three broadcast – to make

a coalition. Hence, the UAVs in a coalition can afford to be the coalition leaders for

the detected targets.

The algorithms developed in this chapter are used to form coalitions and prosecute

targets as and when they are detected in a multiple target scenario where the UAVs

have to search for the targets. Algorithm 4.3 provides an optimal solution to prosecute

a single target. However, it is necessary to know the deviation of the proposed solu-

tion in a multiple target scenario from that of the global optimal solution where the

information about all targets are known a priori. For this purpose, we formulate the

problem as a global static optimization problem and solve it. In the next section, we

will discuss the solution of this combinatorial optimization problem.

4.4 Combinatorial optimization problem

The performance of the mission using the above algorithms – PTCFA and OCFA

– is sub-optimal for an entire search and prosecution mission. In order to know the

deviation of the mission performance of the PTCFA and OCFA algorithms from that of

the global solution obtained using a priori information of the target locations, we need

to solve the global optimization problem of minimizing the sum of prosecution times

for all the targets (refer Eq.( 4.4)). The solution to this combinatorial optimization

problem is NP-hard, in part because it is difficult to capture the dynamics of depletion

the UAV resources after each target prosecution. We will solve the global optimization

problem using the Particle Swarm Optimization (PSO) technique. The PSO allows us

to model the optimization problem taking the resources dynamics into account. Also,

PSO is faster compared to genetic algorithms (Parsopoulos & Vrahatis, 2002). Due

to these features we adopt PSO to solve the optimization problem of minimizing total

114

time to prosecute all the targets, given target locations and resource requirements, and

initial positions and resources of UAVs.

4.4.1 Overview of particle swarm optimization

PSO is a population based stochastic optimization technique developed by Eberhart

and Kennedy (1995), Kennedy and Eberhart (1995). The key idea behind the algorithm

is to simulate the social behavior of bird flocks, fish schools, etc. Each particle in a

swarm is a potential solution in the search space. The particle adjusts its velocity ac-

cording to its own flying experiences and its flock’s experiences. The PSO technique is

similar to the evolutionary computation techniques in (Parsopoulos & Vrahatis, 2002),

however, in PSO each particle can adapt its velocity to move in the search space and

has memory of its best position.

If we assume the optimization problem to be of Q-dimension, then each particle

in the swarm S, at the kth time step, can be represented as Xkl = (xk

l1,xk

l2, . . . ,xk

lQ), l =

1, . . . ,S. The best previous position attained by the particle is represented as Pkl =

(pkl1, pk

l2, . . . , pk

lQ), the velocity of the particle is V k

l = (vkl1,vk

l2, . . . ,vk

lQ), and the best

global position achieved by the swarm is represented as Pkg = (pk

g1, pkg2, . . . , pk

gQ), where

we denote the time step (iteration number) as the superscript k. The particles in the

swarm are updated according to the following equations

vk+1ld

= χ(wvkld + c1r1(pk

ld − xkld )+ c2r2(pk

gd− xkld )), (4.16)

xk+1ld

= xkld + vk+1

ld(4.17)

for d = 1, . . . ,Q, where, r1 and r2 are uniformly distributed random numbers in [0, 1],

c1 and c2 are positive constants representing cognitive and social parameters, w is

the inertia weight, and χ is the constriction factor. The role of inertia weight w is

to create a balance between global and local explorations. Initially, it is necessary to

explore the search space and then reduce w as the solution reaches the optimum value

(Shi & Eberhart, 1998). The constants c1 and c2 aid in convergence of the solution.

The random parameters r1 and r2 are used to maintain the diversity of the swarm

population, while the constriction factor controls the effect of velocity on the particles.

The update equations given in (4.16) and (4.17) are modified from the original

equations developed by Eberhart and Kennedy (1995), which did not have the inertia

and constriction factors. Next, we will determine coalitions for all the targets and

the sequence in which the targets are prosecuted by solving the global optimization

problem using PSO.

115

4.4.2 Solution to the optimal coalition formation problem using PSO

The value of the particle in each dimension can, in general, be a decimal value, but

for the target assignment problem we need it to be an integer. The usual method of

using PSO for integer programming optimization methods is to round the value of the

particle (Laskari et al., 2002; Nemhauser & Wolsey, 1999). we will also use the same

scheme to solve our coalition formation integer programming problem using PSO.

We slightly modify the notation of the particle Xkl to suit our problem. We assume

that N UAVs and M targets are present in the search space. The lth particle at the kth

iteration is

Xkl = (xk

l1,1,xk

l1,2. . . ,xk

l1,M,xk

l2,1,xk

l2,2, . . . ,xk

l2,M, . . . ,xk

lN,1,xk

lN,2, . . . ,xk

lN,M) (4.18)

where, xkli, j∈ {0, . . . ,M} and indicates that UAV Ui will prosecute target xk

li, jin the jth

sequence. A value of 0 for xkli, j

represents a search. The dimension of the particle is

NM. The first M dimensions of the particle present the order in which the first agent

would prosecute different targets. The dimensions M + 1 to 2M present the order in

which the second agent will prosecute targets, and so on.

The PSO algorithm for optimal coalition formation is discussed below. The main

algorithm is given in Algorithm 4.4. The input to the algorithm are the particle swarm,

swarm parameters, termination conditions including the maximum number of itera-

tions (nIterations), and the UAV and target positions and resources. The algorithm will

output the state of best particle in the swarm – particle with least cost. For each iter-

ation, all the particles are evaluated using evaluateSwarm function (line 3) described

in Algorithms 4.4 to 4.4. Before evaluating a particle in this function, it is checked

whether the state of the particle is a feasible one or not (line 19). If not, then the

value of the particle is set to infinity (line 48). A valid particle is evaluated for the

cost of the coalition formation solution suggested by its state (lines 22 to 46). If the

coalition solution does not prosecute all the targets, then the particle is assigned a high

cost (line 45). For each sequence of prosecution, the targets to be prosecuted during

that sequence is identified (line 24). Then, for each of these targets, Tj, we find all

the UAVs (uavsPreferringTarget) that are willing to prosecute Tj during that sequence

(line 28). If a particular target is already assigned in a previous sequence, then the

cost of that particle is set to infinity (line 39). If the combined resources of UAVs in

uavsPreferringTarget is greater than or equal to the target resource requirement, then

a coalition is formed and the coalition cost is added to the cost of the particle (line

31). Else, a coalition is not feasible and the particle is assigned a very high cost (line

36). If the coalition is feasible, then the target Tj is added to the targetsAssigned list,

116

Algorithm 4.4: Global solution using PSO

Input: swarm S, swarm parameters, termination conditions, UAV and target positionsand resources

Output: particle in S with least cost

1: bestParticleValue = ∞, bestParticle = [ ]

2: for kIteration = 1 to nIterations do

3: swarmParticleValues← evaluateSwarm(S)

4: [minParticleValue, minParticleIndex]← min(swarmParticleValues)

5: if minParticleValue ≤ bestParticleValue then

6: bestParticleValue← minParticleValue;

7: bestParticle← S(minParticleIndex);

8: end if

9: if termination conditions are met then

10: return bestParticle

11: break

12: else

13: update S using Eqs. (4.16) and (4.17)

14: roundoff xkli, j

to nearest integers for every particle in S

15: end if

16: end for

117

Algorithm 4.4: (continued) Global solution using PSO

17: Function cost = evaluateSwarm(S)

18: for lParticle = 1 to S do

19: validParticle← if each entry xkli, j

in the sate of the particle belongs to {0,1, . . . ,M}

20: targetsAssigned← [ ]

21: if validParticle then

22: cost(lParticle)← 0

23: for jSequence = 1 to M do

24: targetsInCurrentSequence← S(lParticle,jSequence:M:NM)

25: while targetsInCurrentSequence = /0 do

26: Tj ← targetsInCurrentSequence(1)

27: if Tj /∈ targetsAssigned then

28: uavsPreferringTarget← UAVs with entry as Tj in jSequenceth sequence

29: coalitionResources← sum of resources of uavsPreferringTarget

30: if coalitionResources ≥ RTj then

31: cost(lParticle)← cost(lParticle) + max(ETA(uavsPreferringTarget))

32: targetsAssigned← append Tj

33: reduce the resources of the UAVs in uavsPreferringTarget

34: record UAV-target assignment, coalition time, and prosecution sequence

35: else

36: cost(lParticle)← ∞

37: end if

118

Algorithm 4.4: (continued) Global solution using PSO

38: else


40: end if

41: targetsInCurrentSequence = targetsInCurrentSequence \ Tj

42: end while

43: end for

44: if targetsAssigned = {1, . . . ,M} then


46: end if

47: else


49: end if

50: end for

resources of UAVs in the coalition are depleted, and the UAV-target assignment, coali-

tion time and the sequence of prosecution are recorded (lines 32 to 34). The coalition

time, which is added to the particle cost (line 31) is found as the maximum of ETAs

of all the UAVs in the coalition (uavsPreferringTarget). This ETA to target computed

for a UAV includes the coalition time for the targets that the UAV is assigned to in

any of the previous sequences. Once all the particles are evaluated, we determine that

particle (minParticleIndex) which has the minimum cost (minParticleValue) (line 4).

If minParticleValue is less than the previous best value then the bestParticleValue and

bestParticle variables are updated (lines 5 to 8). If any of the termination conditions

is met, then the algorithm outputs the state of the best particle (line 10). Else, the

swarm is updated using the PSO Eqs. (4.16) and (4.17) (line 13). The algorithm termi-

nates when one of the termination conditions is met or when the maximum number of

specified iterations (nIterations) is reached.

119

Figure 4.5: Initial positions of the UAVs and the targets for the example given toillustrate PSO algorithm.

The PSO algorithm produces the optimal coalition assignment for all the targets. It

minimizes the objective function in Eq. (4.4). However, the dimension of the particle

depends on the number of UAVs and targets. With an increase in the number of UAVs

and targets, the computational time increases as PSO has to search on NM dimensional

solution space.

A hypothetical scenario is considered to demonstrate the execution of the PSO

algorithm. Consider a scenario with 3 targets and 4 UAVs whose initial positions are

given in Fig. 4.5. The resources of the UAVs are RU1 = (1,2,3), RU2 = (2,0,1), RU3 =

(1,3,1), and RU4 = (1,2,1), and the target resource requirements for this example are

RT1 = (2,1,0), RT2 = (1,0,1), and RT3 = (0,1,3).

The solution, Xkl , obtained using the PSO algorithm, after 532 iterations, is (300

020100100). From the solution, according to line 24 of Algorithm 4.4, targetsInCur-

rentSequence = [3011]. This makes T3 the current target Tj (line 26). Since only one

UAV has target T3, and it satisfies the resource requirement, U1 is assigned to T3 (line

34). The distance of the UAV to target using the roundabout path is added to the cost in

line 31. The target is removed from the targetsInCurrentSequence vector which is now

[011] (line 41). The next entry (target) in the solution vector is 0, which is a search,

hence it is not taken into account. Therefore the targetsInCurrentSequence is [11].

The next target is T1, and carrying out procedure similar to before, we can see that the

120

Figure 4.6: The target assignment achieved using PSO in the example given to illus-trate PSO algorithm.

resource constraint is satisfied when U3 and U4 are assigned to T1. The coalition cost is

added to the particle cost and the target is removed from the targetsInCurrentSequence

list. Since the targetsInCurrentSequence list is empty for the current sequence, the next

sequence is considered. For the second sequence, we have targetsInCurrentSequence

= [0200]. Since, target T2 is the only target present in this sequence, it is assigned

to U2 as resource constraints are met. The particle cost is updated by adding the cost

of U2 moving to T2. The remaining sequence has only search task and thus does not

contribute to the cost. Figure 4.6 shows the assignment of the UAVs to targets. In the

figure we can see that U1 is assigned to T3, U3 and U4 are assigned to T1, and U2 is

assigned to T2. The cost given by the PSO solution is the sum of the coalition times

for all the targets. Note that the formulation of the PSO problem would allow multiple

solution vectors giving the same solution with identical cost. For example, a parti-

cle with state (300200100100) also would have given exactly the same UAV-target

assignment and cost as the one given in the example.

In the next section, we conduct extensive simulation studies to evaluate the perfor-

mances of the coalition formation algorithms which we developed in this chapter for

search and prosecute missions.

121

4.5 Simulation results

Monte-Carlo simulations are done to evaluate the performances of the single target

coalition formation algorithms (PTCFA and OCFA) as well as the global optimal coali-

tions for all the targets. The complexity of the optimization problem increases with in-

crease in number of UAVs and targets. We will analyze the performance of algorithms

in terms of the time required to complete the mission (mission completion time), time

required to form a coalition (coalition formation time), and the time required to simu-

late one complete mission with a given target and UAV distribution (simulation time)

for both the two-stage algorithms and the PSO solution.

4.5.1 Solutions obtained using PSO and two-stage algorithms: an example

We consider a sample mission scenario with 4 UAVs and 4 targets and analyze the

time taken by each algorithm to form a coalition, the time taken to accomplish the

mission, and the simulation time (that is, the time the computer takes to simulate a

given mission scenario). The region of interest is an area 1000 m × 1000 m in size,

and the initial position of the UAVs and targets are shown in Figure 4.7. Every UAV is

assumed to have a constant speed of 10 m/s and a sensor range of 300 m. A UAV that

goes out of the 1000 m×1000 m region of interest during its search takes a minimum

radius turn to come back into the region. The available resources of the UAVs are

RU1 = (2,3,4), RU2 = (2,1,3), RU3 = (3,2,4), and RU4 = (2,2,0), and the required

resources of the targets are RT1 = (1,1,2), RT2 = (3,2,4), RT3 = (2,1,2), and RT4 =

(3,4,1).

PTCFA solution

The UAVs have to search for targets and when detected need to form coalitions. As

shown in Fig. 4.8 at time t = 0.6 s, U1 detects target T1 and its resources satisfy the

condition RU1 ≥ RT1 . Hence, the UAV is in state S1 and it prosecutes the target without

sending a coalition request as described in Section 4.2. The trajectory of U1 traveling

toward T1 following a roundabout path is shown in Fig. 4.8. At time t = 1 s, U3 detects

T4 and forms a coalition with UAVs U2 and U4. These UAVs adjust their path and travel

toward T4 which is shown in Fig. 4.10. After prosecuting the assigned targets, the

UAVs continue to search the region. At t = 147.8 s, U3 detects T3 and a coalition with

U4 is formed. These UAVs adjust their path length to meet the rendezvous constraint

122

Figure 4.7: Initial positions of the UAVs and the targets for the given example.

U2

U4

U1

U3

T3

T4

T1

T2

Figure 4.8: The trajectory of U1 prosecuting T1 using PTCFA.

123

U2

U1

U4

U3

T3

T1

T4

T2

Figure 4.9: The trajectory of U1 prosecuting T1 using OCFA.

T3

T4

T2

T1

U2 U3

U4

U1

Figure 4.10: Routes followed by the coalition U2,U3, and U4 formed by PTCFA toprosecute T4.

124

U2

U4

U3

U1

T3

T2

T4

T1

Figure 4.11: Routes followed by the coalition U3 and U4 formed by OCFA to prosecuteT4.

T2

T3

U3

U4 U2

U1

Figure 4.12: UAVs U3 and U4 assigned to T3 by PTCFA.

125

T3

T2

T1

T4

U4

U2

U1

U3

Figure 4.13: UAV U2 assigned to T3 by OCFA.

T2

U3U4

U1

U2

Figure 4.14: Trajectories of UAVs U1 and U2 prosecuting target T2 using PTCFA.

126

T2

U2

U1

U4 U3

Figure 4.15: Trajectories of UAVs U1,U3, and U4 prosecuting target T2 using OCFA.

T1

T3

T2

T4

U2

U3

U1

U4

Figure 4.16: Coalition formation and prosecution of all the targets in the given exampleusing PSO.

127

and their trajectory is shown in Fig. 4.12. During this time, T2 is detected by U1 at

t = 164 s and a coalition with U2 is formed. The routes taken by U1 and U2 are shown

in Fig. 4.14. The total mission was accomplished in 263 seconds. Next, we execute

the same mission using OCFA, which will demonstrate that the coalition formed using

PTCFA is sub-optimal.

OCFA solution

In the mission using OCFA, U1 detects T1 at t = 0.6 s and it assigns itself to T1 as the

resource requirement is met. The route followed by U1 to T1 is shown in Fig. 4.9. At

t = 1 s, U3 detects T4 and, unlike PTCFA, generates a coalition along with U4 only (the

coalition U3 generated using PTCFA produced a coalition with 3 UAVs). Figure 4.11

shows the trajectory of UAVs U3 and U4 traveling to T4. UAV U2 detects target T3 at

t = 1.4 s and since it satisfies the target resource constraint and is not assigned to any

target, the state of U2 is S1 and thus produces a single member coalition. The route

taken by U2 to T3 is shown in Fig. 4.13. The final target T2 is detected by U1 at time

t = 155 s and a coalition with UAVs U2,U3, and U4 is formed. The trajectories of these

UAVs to target is shown in Fig. 4.15. The mission is accomplished in 213.8 s using

OCFA.

PSO solution

When the target positions and their resources are known a priori, we can solve the

combinatorial optimization problem of coalition formation using PSO. Figure 4.16

shows the assignment and the routes that the UAVs take to accomplish the mission.

UAV U1 is initially assigned to T1 and then to T4 in the next sequence, while U4 is

assigned to T4 only. So, U4 takes a path, the length of which is equal to the sum of path

length of U1 prosecuting target T1 and the length of the roundabout path of U1 from

T1 to T4. The entire mission is accomplished in 114.3 seconds using the assignment

determined by the PSO. The optimal solution is obtained within 600 iterations.

Comparison of performance of coalition formation algorithms

Table 4.3 summarizes the mission time and simulation time for a mission using PSO

solution and the two-stage coalition formation algorithms. The simulations were car-

ried out using MATLAB on a 1.83 GHz, 1 GB RAM machine. The performance of the

mission using PTCFA is poorer than that achieved using OCFA in terms of the mission

128

Table 4.3: Comparison of the mission and simulation times obtained with the PSOsolution and the two-stage algorithms

Algorithm Mission time (s) Simulation time (s)PSO 114.3 17.3PTCFA 263.0 3.31OCFA 213.8 3.83

completion time, because the number of UAVs performing search task is reduced in

PTCFA as the coalitions are not of minimal size and hence the search to detect targets

is less efficient. Although the performance of the mission using OCFA is better, the

computational time taken is higher than that for the PTCFA algorithm.

From the table we can see that the mission can be accomplished in much lesser

time using the PSO solution than using any of the two-stage algorithms as the target

positions are known a priori in this case. However, the simulation time taken to com-

plete the mission using the PSO is much higher than any of the two-stage algorithms.

In case of PSO solution, the time taken to produce a solution increases exponentially

as we increase the number of UAVs and targets. This effect is studied in the next

sub-section.

Another advantage of using two-stage algorithms that we developed is that it is

decentralized and can take uncertainties in the environment into account, which may

include incomplete knowledge of resources and positions of the targets and the UAVs.

4.5.2 Effect of increase in number of UAVs and targets

A set of 100 simulations are carried out using PTCFA, OCFA and PSO solutions on

a 1000 m × 1000 m area. For the simulations using PTCFA and OCFA, the parame-

ters for the UAVs are a constant speed of 10 m/s, minimum turning radius of 50 m, a

sensor range of 100 m, and a simulation time of 1000 s. The simulation time of 1000 s

signifies that the mission is abandoned after 1000 s even if all the targets could not be

prosecuted within this time. The simulation parameters for the PSO coalition forma-

tion algorithm are c1 = 0.5,c2 = 0.5,χ = 1,w ∈ [0.4 0.7], and the maximum number

of iterations = 5000. For each simulation, the target positions and resources, and the

UAV positions and resources are randomly generated. For each target, the quantity

of each type of resource required for its prosecution is chosen as a random integer

between 0 and 3. We allocated the quantity of resources for the UAVs based on the

number of available targets. Thus, each type of UAV resource is a random integer be-

129

5 10 15 200

100

200

300

400

500

600

700

Number of UAVs

Ave

rage

mis

sion

com

plet

ion

time

PTCFAOCFAPSO solution

Figure 4.17: Comparison of the mean mission completion times in seconds for a mis-sion with 5 targets plotted against the number of UAVs in the mission.

tween 0 and (number of targets)/2. A UAV can carry a maximum of 3 different types

of resources. For a given number of targets, we study the mission performance using

various coalition formation algorithms for varying the number of UAVs – 5, 10, 15 and

20. We consider cases where the number of targets are 5, 10, and 15. We also study

the effect of percentage mission accomplished, time taken to form the coalition using

PTCFA and OCFA (coalition formation time), and the CPU time required to carry out

each simulation (simulation time).

The resources for the UAVs and the targets are generated randomly as described

above. The simulations are carried out in different ways for the two-stage and the

PSO solutions. For the simulations using PTCFA and OCFA, the total time required

to prosecute all the targets is recorded as the mission time. If some of the targets

are not prosecuted within the simulation time, either due to lack of UAV resources

or because they were not found during the search, then the percentage of the mission

accomplished is calculated as

% mission completed =Number of target prosecuted

Total number of targets×100.

The UAV team does not have a priori knowledge of whether the cumulative resources

of the team are sufficient to prosecute all the targets or not. If the resources are not

sufficient to prosecute all the targets, then the UAVs continue the search task till the

end of simulation time, that is 1000 s, else, the time taken to prosecute all the targets is

130

5 10 15 2050

55

60

65

70

75

80

85

90

95

100

Number of UAVs

Ave

rage

per

cent

age

mis

sion

com

plet

ed


Figure 4.18: Mean percentage mission not accomplished due to the lack of sufficientresources for a mission with 5 targets plotted against the number of UAVs in the mis-sion.

5 10 15 200

100

200

300

400

500

600

700

800

900

1000

Number of UAVs

Ave

rage

mis

sion

com

plet

ion

time


Figure 4.19: Comparison of mean mission completion times in seconds for a missionwith 10 targets plotted against the number of UAVs in the mission.

131

5 10 15 200

10

20

30

40

50

60

70

80

90

100

Number of UAVs

Ave

rage

per

cent

age

mis

sion

com

plet

ed


Figure 4.20: Mean percentage mission not accomplished due to the lack of sufficientresources for a mission with 10 targets plotted against the number of UAVs in themission.

5 10 15 200

100

200

300

400

500

600

700

800

900

1000

Number of UAVs

Ave

rage

mis

sion

com

plet

ion

time


Figure 4.21: Comparison of mean mission completion times in seconds for a missionwith 15 targets plotted against the number of UAVs used for the mission.

132

5 10 15 200

10

20

30

40

50

60

70

80

90

100

Number of UAVs

Ave

rage

per

cent

age

mis

sion

com

plet

ed


Figure 4.22: Mean percentage mission not accomplished due to the lack of sufficientresources for a mission with 15 targets plotted against the number of UAVs used forthe mission.

5 10 15 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Number of UAVs

Ave

rage

coa

litio

n tim

e in

mill

i sec

OCFA

PTCFA

Figure 4.23: Average computational time taken to form coalitions using PTCFA andOCFA.

133

5 10 15 200

500

1000

1500

2000

2500

3000

3500

4000

Number of UAVs

Ave

rage

sim

ulat

ion

time

in s

ec

5 target10 targets15 targets

Figure 4.24: Mean time taken to complete a simulation using PSO.

recorded as mission time. We also record the time to form the coalition and the CPU

time needed to perform the simulation.

For the PSO based optimal solution, each simulation was carried out by running

the PSO algorithm for a given number of iterations (5000) and the mission time ob-

tained from the solution and the CPU time taken for the solution are stored. When the

cumulative UAV resources are less than the cumulative target resources then a solution

using PSO cannot be obtained and the percentage mission accomplished is 0%, since

a feasible solution could not be found, and the mission time in that case is treated as

1000 s.

The performance curves for the Monte-Carlo simulations are shown in Figs. 4.17–

4.22. Let us first consider a mission with 5 targets and varying number of UAVs as

shown in Fig. 4.17. From the figure, we can see that the optimal solution obtained

from PSO is better than the solutions obtained using two-stage algorithms. However,

the deviation of the performance by PTCFA and OCFA algorithms from that of the

optimal is marginal. This shows that the algorithms that we developed are not far from

being optimal. The time to accomplish a mission has two components (i) the search

time to detect the target and (ii) the time to prosecute the target. If the UAVs have small

sensor range, then they will take more time to detect all the targets. Also, the target

detection time depends on the kind of search strategy used by the UAVs. Because of

134

the lower sensor range and randomized search strategy, the mission completion time is

higher for the two-stage algorithms. The mission time using these algorithms can be

reduced by increasing the sensor range and also using a better search scheme that can

detect targets early.

Figure 4.18 shows the percentage of the missions that were completed. From the

figure we observe that the percentage mission completed when we use PSO solution

is low for 5 UAVs compared to that obtained using the PTCFA and OCFA algorithms.

This is because for some of the Monte Carlo runs, the UAVs did not have sufficient

cumulative resources to prosecute all the targets. In those cases the problem does

not have an optimal solution leading to zero percent mission completion, and hence

decrease in the percentage mission completed. But, for PTCFA or OCFA algorithms,

as the targets are prosecuted when detected, the percentage mission accomplished is

higher than that of the PSO solution. When the number of UAVs are increased to 10,

15 and 20, all the 5 targets are prosecuted by the UAVs, therefore the mean percentage

mission completed is 100% for all the algorithms.

The mission performance for the case of 10 targets is shown in Fig. 4.19. For the

mission with 5 UAVs, 99% of the simulations did not have sufficient UAV resources to

prosecute all the targets. Hence, the average time for the mission is high. However, for

the simulations with higher number of UAVs, the mission time decreases. The PSO

solutions are better than the PTCFA and OCFA solutions. Although the PSO solution

is better in terms of mission completion time, the percentage mission accomplished is

lower than that achieved using PTCFA and OCFA algorithms due to the reason that we

gave earlier. Similar behaviors in the performances of algorithms can also be seen for

mission with 15 targets and is given in Figs. 4.21 and 4.22.

From these simulations we can conclude that even though the PTCFA produces

sub-optimal coalitions, its performance is comparable to that achieved using OCFA

and PSO solution. Thus, owing to its polynomial time complexity and hence scalabil-

ity, PTCFA stands as a good candidate for a real-time coalition formation algorithm to

be implemented in a large scale search and prosecution mission.

When a coalition leader receives proposals from potential coalition members, it

uses PTCFA or OCFA to determine the coalition for the task. Figure 4.23 shows the

mean computational time taken to form coalitions for various target and UAV distri-

butions using these algorithms. From the figure we can see that the time taken for

OCFA is higher than that using PTCFA for any given UAV and target distribution.

However, the time taken to solve these algorithms is less than one millisecond for both

the algorithms which is encouraging for real-time applications.

135

With increase in the number of UAVs or targets, the simulation time increases

when the PSO algorithm is used. Figure 4.24 shows the average time taken (with

standard deviation bars), to complete one simulation using PSO. For a given number

of targets, the total number of computations increases with increasing number of UAVs

and hence the time to determine the optimal solution also increases.

4.5.3 Coalition formation for UAVs with heterogenous speed

In all the previous simulations, we assumed that the UAVs have same constant speed.

However, we can consider UAVs with different speeds without changing any part of

the algorithm. This is because the algorithm uses ETA of the UAVs which implicitly

captures their velocity information. We present an example with five UAVs and three

targets. The initial locations of the UAVs and the targets are shown in Fig. 4.25. The

UAVs use the PTCFA algorithm to determine coalitions. The UAV speeds are v1 =

8 m/s, v2 = 10 m/s, v3 = 13 m/s, v4 = 15 m/s, and v5 = 18 m/s. The UAV resources

are RU1 = (1,1,1), RU2 = (2,1,1), RU3 = (2,2,1), RU4 = (2,2,1), and RU5 = (1,1,1),

and the target resources are RT1 = (1,3,1), RT2 = (3,3,1), and RT3 = (3,1,3). The

simulation environment is the same as described in Section 4.5.2.

At time t = 0.4 s, the UAV U3 detects target T1 and forms a coalition of UAVs U4

and U5. The trajectories followed by U4 and U5 to prosecute T1 is shown in Fig. 4.26.

As the mission progress, UAV U2 detects target T3 and forms a coalition consisting of

U1,U2 and U3 at time t = 23.1 s. The trajectories of these UAVs to T3 are shown in

Fig. 4.27. The UAVs U4 and U5 prosecute T1 at t = 27.4 s and subsequently they pros-

ecute T3 at time t = 98.6 s. The final target T2 is detected by U3 and forms a coalition

of U1,U3,U4, and U5 at time t = 123 s. The target is prosecuted at time t = 198.2 s and

the trajectories of the UAVs are shown in Fig. 4.28. From the trajectories, we can see

that even though the initial ETAs of the UAVs are different, the simultaneous strike

mechanism ensures that all the UAVs reach the target at the specified time which is the

maximum of ETAs of all UAVs in the coalition.

4.5.4 Discussion

The simulation results show that the combinatorial optimization solution using PSO

provides the lowest mission completion time. However, if there is an uncertainty in

the number of resources allocated to UAV or in the information about the targets, then

the solution obtained using combinatorial optimization is not valid any more. The

advantages of forming coalitions in a decentralized manner is that the UAVs need not

136

T1

T2

T3

U1

U2U3

U4

U5

Figure 4.25: Initial UAV and target locations for the given example.

T1

T2

T3

U1

U2U3

U4

U5

Figure 4.26: Trajectories of UAVs U4 and U5 to target T1.

137

T1

T2

T3

U1

U2

U3

U4U5

Figure 4.27: Trajectories of UAVs U1,U2, and U3 to T3.

T2

U1

U2U3

U4

U5

Figure 4.28: Trajectories of UAVs U1,U3,U4, and U5 to T2.

138

have any a priori information about other members in the team or the targets. The

information is required only while forming coalitions. Thus, even if the resources of

some of the UAVs are exhausted or cannot be used for some reason, the decentralized

coalition formation algorithms can handle these uncertainties and still form feasible

coalitions.

Minimum member coalition

The coalition formation algorithm, OCFA, can be modified by considering only a

single objective of minimizing the number of UAVs in a coalition. The time taken to

prosecute the target is not taken into account, in this case. This can be achieved by

removing the first stage of the two-stage algorithms and solving only the second stage

of OFCA.

Improving the performance of PTCFA

The second stage of the polynomial time algorithm attempts to determine the mini-

mum number of agents required for the coalition through a systematic pruning of the

coalition formed during stage 1 (refer Algorithm 4.2). However, this algorithm is naive

and we can incorporate different mechanisms that can provide better solutions. A pos-

sible mechanism is to sort the UAVs by the resources and then screen them, in which

case, the UAVs with low resources will be pruned from the coalition that might result

in a coalition with lesser members. However, there is no guarantee that even such a

mechanism would produce an optimal member coalition.

Other applications

The coalition formation algorithm can be used for many other applications where a

sub-team of UAVs are required to perform a task. These include cooperative tracking

of a suspect on the freeways and in urban environments, patrolling sea shores, and

cooperatively tracking and prosecuting moving targets in a battle-field. In these appli-

cations, the coalitions are formed to share sensor information and the resources may

not deplete. The proposed coalition formation algorithms can be easily modified de-

pending on the mission. For example, consider a cooperative target tracking mission

where the coalition leader detects a target and assigns a coalition of UAVs to track the

target with varying sensors on board and prosecute it (Kingston & Schumacher, 2005).

In this case, the UAV with the prosecute capabilities can have depleting resources.

139


In this chapter, we addressed the minimum time minimum member coalition forma-

tion problem for a search and prosecute task, and showed that the problem is NP-hard.

Then we presented two decentralized two-stage coalition formation algorithms. The

first one has polynomial time complexity and the second uses binary integer program-

ming technique with low computational overhead. The algorithms determine a time

optimal set of agents that have sufficient resources to simultaneously prosecute a target.

In order to determine the deviation of the solution provided by the proposed algorithms

from an optimal solution, we developed a PSO formulation of the target prosecution

problem to solve the centralized combinatorial optimization problem. Simulation re-

sults showed that the mission performances of the two-stage algorithms are close to

the PSO solution. The mission times taken by the proposed algorithms, as well as the

time to form coalitions, are less and this holds promise for real-time applications.

This chapter assumed global communication, that is, every UAV can communicate

with every other UAV, which is seldom the case in a realistic UAV mission. Coalition

formation of UAVs with limited communication ranges adds an extra layer of com-

plexity to the problem. This is because the coalition, in this case, has to be formed

over a time varying network produced by UAVs with limited communication ranges in

motion. In the next chapter, we consider this problem and propose a communication

protocol for effective coalition formations over dynamic networks.

140

5 Coalition formation with limited communication

This chapter relaxes the assumption of global communication that was central tocoalition formation algorithm developed in Chapter 4. Also, the prosection of con-stant velocity and maneuvering targets, in addition to that of stationary targets, isconsidered. UAVs with limited communication ranges moving around in search oftargets form a time varying network. Forming coalitions over a dynamic networkis challenging. This chapter develops a mechanism to find potential coalition mem-bers over a network with time varying topology using principles from internet pro-tocol. Simulation results of specific cases are presented that show how coalitions areformed over a dynamic network with moving targets and then Monte-Carlo simu-lations are carried out to study the effects of number of UAVs and targets, and thecommunication parameters on the coalition formation process.

�n this chapter, we make the search and prosecute task considered in Chap-

ter 4 more realistic by imposing the following realistic conditions (i) UAVs

have limited communication ranges and (ii) the targets are not necessarily

stationary. The addition of the condition of limited communication range makes the

coalition formation problem complex. This is because, the limited communication

paradigm, along with the fact that UAVs are in a state of constant motion, gives rise to

a time varying communication network. It is over this changing network that the UAVs

have to communicate to form feasible coalitions to prosecute the detected targets. We

use ideas from internet protocol to solve this problem.

To form a coalition, the request from the coalition leader, seeking participation of

other UAVs in the coalition, should first reach the rest of the UAVs. The coalition

request has to propagate over the network formed by the UAVs with limited communi-

cation ranges. Some of the UAVs receive the coalition proposal, not directly from the

coalition leader but through some intermediate UAVs which propagate (re-broadcast)

the message of the coalition leader. These intermediate UAVs are referred to as relay

nodes. Each re-broadcast will cause a delay in the information propagation which may

result in different UAVs getting the same information at different times.

Since the network is dynamic, finding potential coalition members is difficult as

the network relay nodes (other UAVs) may go out of each others’ communication

ranges disrupting the communication network which is essential to form coalitions.

Apart from this, the earliest time to arrive (ETA) at the target by a UAV depends on

its position which may change considerably by the time the coalition leader makes a

141

TargetsUAVs

T

Ui

j

2

3

4

5

6

7

UU

U

U

U

U

Figure 5.1: Search and prosecute mission using UAVs with limited sensor and com-munication ranges – a sample scenario.

decision about formation of a suitable coalition (coalition formation process takes time

due to communication delays involved). Also, the ETA varies with time for a moving

target which is an important consideration if a rendezvous at target is to be achieved for

prosecution. The coalition formation algorithm should take into account these facts.

In this chapter, we develop a solution that addresses the above issues. This chap-

ter is organized as follows. After formulating the problem in Section 5.1, we propose

a communication protocol, using concepts from internet protocol, that will enable a

coalition leader to find potential coalition members, and form coalitions over a time

varying dynamic communication network in Section 5.2. We discuss, in Section 5.3,

the computation of ETA, and in Section 5.4, the prosecution procedure, for various tar-

gets – stationary, constant velocity, and maneuvering. Section 5.5 gives the simulation

results to show the efficacy of the proposed solution. Section 5.6 gives a summary and

concludes the chapter.

5.1 Problem Formulation

The search and prosecute mission using multiple UAVs with limited communication

and sensor ranges, and limited resources is described below.

5.1.1 The mission

We consider a search and prosecute mission, as in Section 4.1, on a bounded region

142

Figure 5.2: Illustration of why Rcomm should be greater than 2Rsen.

that contains M targets whose initial positions are unknown, using N UAVs with lim-

ited communication and sensor ranges, and limited resources (refer Fig. 5.1). The

objective is to prosecute all the targets in minimum time. As in Chapter 4, the UAVs

are heterogeneous because they carry different types of resources in varying quantities.

Also, the UAVs can have different but constant speeds. The UAVs carry out a search

task to locate targets using their sensors that have a limited sensor range of Rsen. When

Ui detects a target Tj, as shown in Fig. 5.1, we assume that it can ascertain the type of

resources required to prosecute the target.

Unlike the case considered in Chapter 4, each UAV has a limited communication

range (Rcomm) and can communicate with only those UAVs that are within this range.

We also assume that the communication range of a UAV is at least twice as large as

its sensor range, that is, Rcomm > 2Rsen. This assumption is introduced because if the

communication range is smaller than 2Rsen, then multiple UAVs that are not within

the communication ranges of each other can detect the same target resulting in the

formation of multiple coalitions for the same target. This is illustrated in Fig 5.2. For

sake of clarity, communication ranges are shown in the figure for only one UAV.

When a UAV Ui detects and identifies a target Tj, it initiates a coalition formation

process as described in the flowchart in Fig. 4.2 of Chapter 4. However, with the

assumption of limited communication ranges for UAVs, this process has to be carried

out over a time varying network. The issue of determining the potential coalition

members for a coalition through the dynamic network is addressed in Section 5.2.

Recall from Section 4.2 that the coalition leader forms a coalition to prosecute the

target (i) in minimum time, (ii) using minimum number of UAVs, (iii) simultaneously.

143

The objective (i) is desirable as it minimizes the fuel usage during a specific prosecu-

tion, the objective (ii) ensures that the UAVs distribute their search effort so that the

targets are detected quickly and the mission of finding and prosecuting all the targets

is accomplished quickly, and the objective (iii) is essential to inflict maximum damage

to target.

In this chapter, we consider stationary, constant velocity non-maneuvering, and

maneuvering targets. The objective of prosecuting the detected target in minimum

time depends on the correct computation of Earliest Time of Arrival (ETA) to the

target. This is dealt with in Section 5.3. In the case of maneuvering targets, it is

difficult to achieve the objective of simultaneous strike with constant velocity UAVs,

and therefore, is relaxed. The prosecution sequence that we follow for various targets

is given in Section 5.4.

5.1.2 Target and UAV kinematics

We consider the following target kinematics. For a non-maneuvering constant velocity

target Tj with a constant heading ψTj , we have

xTj = VTj cosψTj

yTj = VTj sinψTj

where,(xTj ,yTj

)is the position of Tj and VTj is its velocity.

The kinematics of a maneuvering target, considered in this chapter, is as follows.

A maneuvering target Tj moves straight at its initial speed VTj for a short random time

interval, the length of which is bounded. At the end of this time period, the target

takes a turn randomly either to the left or to the right with a specified turn radius for

a random time interval. After the maneuver, the target goes straight again for another

short period of time and the process repeats.

The UAV kinematics is same as that considered in Chapter 4 (Eqs (4.7) and (4.8)).

We assume that the velocities of all the targets are less than the velocities of any of the

UAVs. Since the mission is assumed to be over a bounded region, when a target reaches

the boundary it makes a random maneuver to enter back into the boundary. Since the

random maneuver that a target might employ at the boundary is not predictable, the

coalition leader forms a coalition only if the target can be prosecuted when it is well

within the boundary. A boundary is defined to keep the simulations tractable and not

due to any limitation of the coalition formation algorithm itself. Also,in a real world

search and prosecute mission, there will always be a region of interest, the targets

beyond which is not of value for the mission.

144

5.2 Communication protocol for coalition formation

The UAVs need to form coalitions over a time varying network and the coalition

formation process depends heavily on the communication protocol employed. The

coalition formation process involves the following communications (refer Fig. 4.2) (i)

request of the coalition leader seeking the participation of other UAVs in the coalition

to be formed to prosecute a target that it detected, (ii) response from potential coalition

members about their willingness to participate in that coalition, and (iii) message from

coalition leader about the success/failure of the coalition formation process. When

the UAVs have limited communication ranges, these communications have to be done

over a time varying network. We address this problem by trying to find a sub-network,

that is invariant for the time period of the coalition formation process, over which the

communications happen in an uninterrupted manner.

Although the coalition formation process we developed in Section 4.2 is simple,

when implemented over a dynamic network, like the one we consider in this chapter,

highly non-trivial situations arise which need to be addressed effectively. For example,

an intermediate UAV can fail to communicate a decision to a required UAV or by the

time the coalition decision is declared, a potential coalition member can be out of

communication range, thus not getting a confirmation about the success/failure status

of the coalition formation process which leads to uncertainty about its membership in

the coalition. To address such situations, we need to design behaviors that will either

enable the agent to make its decision consistently whenever such kind of events occur,

or avoid these situations altogether. We adopt the second strategy, and is described in

detail in this section.

5.2.1 Broadcast in a limited communication network

Like in Chapter 4, we use the broadcast of messages as the mechanism of commu-

nication between the UAVs. In the network formed by UAVs with limited commu-

nication ranges, a UAV will receive information from another UAV (say, a coalition

leader) through intermediate UAVs called relay nodes. For example, in Fig. 5.1, the

UAV U7 receives information about target Tj from Ui through the information path

Ui→U2→U3→U7, where the UAVs U2 and U3 act as the relay nodes re-broadcasting

the message from Ui. This method of communication between wireless nodes is re-

ferred to in the literature as ‘flooding’ (Obraczka et al., 2001; Williams & Camp,

145

2002). To mimic a real-world wireless network, we associate a delay with each mes-

sage hop – transmission of message from one UAV to another – called the hop delay

denoted by Δhop. The hop delay, as we consider it, includes the queuing delay, the mes-

sage processing time at a relay node, and the propagation delay, which are the usual

delays associated with a standard wireless network. The wireless communication stan-

dard – the IEEE 802.11 (Medium Access Control) MAC protocol – handles broadcast

packets by sending it to all the neighbors as soon as the carrier is sensed free (IEEE-SA

Standards Board, 2007). However, it does not use any collision detection to ensure the

successful delivery of the broadcasted packets. There are some schemes (Alagar et al.,

1995; Tang & Gerla, 2001; Chen & Huang, 2005) that ensure reliable MAC designs

for broadcast transmission but add overhead to the MAC layer. We assume that the

UAVs use one such protocol and the hop delay will include the associated overheads

also. The value of Δhop can be considered a constant taking the value of the estimated

maximum possible delay in the network (Lamport, 1984). For the example in Fig. 5.1,

the UAV U7 would receive the message from Ui within 3Δhop seconds as the message

from Ui has to make 3 hops to reach U7.

A message broadcast by a UAV flows through the network as it is re-broadcast by

relay nodes. To prevent the message from floating around in the network indefinitely,

thus reducing the bandwidth of the network, we use the notion of time-to-live (TTL) for

a message – a notion used in packet switching (Clark, 2003) in the Internet Protocol.

We consider the TTL of a message to be the maximum number of hops that it is

allowed to make before it is abandoned (not re-broadcast) by the nodes in the network

and specify it in terms of maximum number of hops (Hmax) a message can make in a

network. The TTL concept is implemented as follows. Each message has in its body a

hop counter, H, which is initially set to Hmax. Whenever a relay UAV re-broadcast the

message, H is reset to H−1. If H = 0, then the message is not re-broadcast by a relay

UAV and thus gets abandoned.

A UAV that acts as a relay node keeps a log of every message that it re-broadcasts.

If it receives a message that is already in the log, it will not be re-broadcast. This is to

prevent clogging the network with messages that are already broadcast. For example,

in Fig. 5.1, U2 gets a message from Ui which it broadcasts. But, it may receive the

same message from U6 later. However, this will be ignored by U2 as it has already

broadcast that message. The log of a broadcast message is kept until the TTL of the

message.

146

5.2.2 Coalition formation over a dynamic network

To determine a coalition, the coalition leader requires the following information from

the potential members: (a) earliest time of arrival (ETA) at the target, and (b) resources

available for target prosecution. However, in the present case, since the target is non-

stationary and the network has delays, there is an ambiguity about the procedure for

calculation of ETA. In order to determine ETA in a consistent manner, we draw in-

spiration from the way contracts are carried out for tasks in real life. An excellent

example of this can be the task of ‘call for papers’ for a conference. The conference

committee will broadcast the task with the submission deadline, the deadline for ac-

ceptance/rejection notification, and the final submission deadline. Thus, there exists

deadlines for every part of the process to accomplish the task. Similarly, the coalition

leader uses intermediate deadlines to receive the information and to form coalitions.

The coalition leader will broadcast the availability of the target along with the deadline

for proposal submission, the deadline to receive decisions, and the deadline to pros-

ecute the target. The coalition leader will also broadcast the exact time at which the

coalition members are required to start the execution maneuver. The time at which a

coalition member is expected to start its prosecution maneuver is called the go-ahead

time, denoted by τTj

G . A potential coalition member uses this information to estimate

its ETA to the target; ETA being the earliest time to arrive at the target since τTj

G . This

is described in detail in the next section.

We circumvent the problem of forming a coalition over a time varying network

by finding a sub-network that is invariant during the coalition formation process. This

is achieved as follows. A UAV agrees to be a relay node for a message it received

only if it is going to be in the communication range of the UAV from which it re-

ceived the message for the entire coalition formation period. Every message contains

the information on the latest time by which the coalition process should end. For a

coalition formation process that was initiated to prosecute target Tj, this is the same as

the go-ahead time τTj

G , determination of which is described below.

The go-ahead time is determined by the coalition leader as a time that is greater

than the time required for the entire coalition formation process. If Hmax is the maxi-

mum number of allowed hops for a message with a hop delay of Δhop associated with

each hop, then the coalition request, the response to this request, and the message on

coalition membership together takes at most 3HmaxΔhop seconds to propagate over the

network. Each potential coalition member is given a time window of Δw seconds be-

fore they respond to a coalition participation request from a coalition leader. This will

147

give the potential coalition members time to consolidate the requests and respond to the

appropriate coalition leader (coalition leader with highest token number as discussed

in Section 4.2). The coalition leader takes a time of Δc seconds to form a feasible coali-

tion. Thus the total coalition process is upper bounded by 3HmaxΔhop + Δw + Δc. The

coalition leader, therefore, can choose the go-ahead time, τTj

G as 3HmaxΔhop +Δw +Δc.

From the above discussion, it is clear that the time for proposal submission by

potential coalition members is 2HmaxΔhop +Δw seconds after the dispatch of the coali-

tion participation request. A potential coalition member shows interest in a coalition

only if it is in the communication range of the UAV from which it received the mes-

sage until τTj

G . This behavior ensures a sub-network that is invariant over the period

of coalition formation. A potential coalition member Uk, if has any resource required

to prosecute the target, estimates its position at time τTj

G – called its go-ahead loca-

tion, denoted by GTjUk

. It then broadcasts its ETA to the target from GTjUk

along with

the information on its resource capabilities. The coalition leader determines a feasible

coalition using PTCFA (cf. Section4.2.1), OCFA (cf. Section4.2.2), or the PSO solu-

tion (cf. Section 4.4) and the message is broadcast. In case where a simultaneous strike

is desirable, the message is broadcast along with the information on time to intercept

the target. If a coalition leader cannot form a feasible coalition for a target, it waits

for a specified time (a value of 5 s is used in simulations) before a coalition formation

request is broadcast again. The UAVs in coalition reach their respective go-ahead loca-

tions (GTjUk

) and proceed to prosecute the target. On the other hand, the rejected UAVs

continue their previous task.

The above mechanism assumes that all the UAVs have synchronized clocks. If

the time varying communication graph of the UAVs are periodically jointly connected,

which will be the case as the UAVs operate in a bounded region, then the synchroniza-

tion of internal clocks can be achieved.

5.2.3 Structure of the broadcast messages

We give the structure of various messages – coalition participation request, response

to the coalition participation request, and confirmation of coalition membership – that

are broadcast in the network. Coalition leader Ui broadcasts the request in the form

〈Ui,Tj,τTj

G ,H,Hmax〉

where, τTj

G is the go-ahead time, Tj represents the relevant information about the target,

and H is the hop counter which is set to Hmax by the coalition leader. The presence

148

of Hmax in the message is to aid the potential coalition member to initialize the hop

counter H for the proposal message that it is sending back to the coalition leader.

The information on τTj

G , apart from being useful for a potential coalition member to

calculate its ETA to target in a consistent way, enables the intermediate UAVs to decide

whether to be relay nodes and potential coalition members. The go-ahead time also

helps the relay nodes in deciding when to delete the message from its log of sent-

messages in view of saving space.

A potential coalition member, if it has at least one of the required resources to

prosecute a target, will respond to the coalition participation request by a coalition

leader. A typical response of a potential coalition member, Uk, is of the form

〈Uk,Ui,RUk ,τTj

Uk,τTj

G ,H〉

where, RUk is the resource vector of Uk, τTjUk

is its ETA to target, and H is the hop

counter for the message. Similar to the case of broadcast from the coalition leader, the

τTj

G information will help an intermediate UAV to decide whether to be a relay node.

The message send by the coalition leader Ui after a coalition is formed is of the

form

〈Ui,C (Ui,Tj),τTj

G ,H〉

Here, the coalition set C (Ui,Tj) is empty if a successful coalition cannot be formed.

The decision process and the communication protocol during a coalition forma-

tion process, within a group of UAVs with limited communication ranges executing

a search and prosecute task, is given in Fig. 5.3. Although, the coalition leader, the

relay nodes, and the potential coalition members are shown separately in the figure for

clarity, every UAV, including the coalition leader, acts as both a relay node as well as

a potential coalition member.

5.3 Determining ETA to target

A coalition leader needs the information of ETAs of the potential coalition members

to determine a feasible coalition. How a potential coalition member calculates its ETA

to the target, when the target is stationary, non-stationary and non-maneuvering, is

described below.

149

Uk w

ithin

com

m. r

ange

of

rela

y U

AV

?

Uk ha

s any

re

sour

ces

for T

j?

Cal

cula

te t

T j

and

broa

dcas

t alo

ngw

ith R

Uk

Ui

In c

oalit

ion?

Det

arm

ine

path

to ta

rget

and

pros

ecut

e it

No

coal

ition

inte

rest

Mes

sage

in

log

orH

=0?

Ul 1 w

ithin

com

m. r

ange

of

broa

dcas

ting

UA

V?

Re-

broa

dcas

t mes

sage

with

H

H-1

Log

mes

sage

No

broa

dcas

t

Bro

adca

st <

Ui,T

j,t T j

,H,H

max

>G

Rec

eive

pro

posa

lsfo

r coa

litio

ns

Coa

litio

nfe

asib

le?

Det

erm

ine

and

broa

dcas

tt

T j a

nd C

(Ui,T

j)c

Bro

adca

st'd

isca

rd c

oalit

ion'

Mes

sage

from

coa

litio

n le

ader

Mes

sage

to c

oalit

ion

lead

er

Coa

litio

n le

ader

Ui

Rel

ay n

ode

Ul 1

Rel

ay n

ode

Ul n

Pote

ntia

l coa

litio

n m

embe

r Uk

No

Yes

Yes

No

No

No

NoNo

Yes

Yes

Yes

Yes

Figure 5.3: Decision process and communication protocol for coalition formation ofUAVs with limited communication ranges.

150

5.3.1 Stationary targets

The determination of ETA to a stationary target is same as described in Section 4.3;

we compute the ‘roundabout Dubins path length’ to the target.

5.3.2 Constant velocity targets

Similar to the case of stationary targets, the length of the actual Dubins path to target

is not a consistent measure to calculate ETA for the case of moving target too. This is

illustrated in Fig. 5.4. In Fig. 5.4, UAV U1 has a lower Dubins path length to target as

it can intercept the target at point 1 while the earliest interception point for U2 is point

2. Now, if U1 and U2 form a coalition, then U1 has to adjust its trajectory to intercept

the target at point 2 so as to achieve a simultaneous strike, which, as clearly seen in

Fig. 5.4(a), is not possible if it is to follow a Dubins path. Hence we use the roundabout

path as shown in Fig. 5.4(b) to the target. As shown in the figure, U1, in this case, can

take a path with a higher turn radius and hence a higher path length to intercept the

target at point 2 – the dotted path in the figure. Since the target is moving, we employ

an iterative technique to calculate a UAV’s ETA to target using its roundabout path

length to target. This is similar to an algorithm, referred to as MADDOG (Reid et al.,

1981), found in the missile guidance literature and is explained below.

The potential coalition member Uk, from the message of the coalition leader, knows

the position (ITj

0 ) and velocity of the target at time τTj

G . Also, the UAV Uk can predict

its own location at τTj

G denoted by GTjUk

. Let ITj

B be the point where the target crosses the

boundary (refer Fig. 5.5). Note that a boundary is defined to demarcate the region of

interest; a target out of the boundary is considered not of interest for the multiple UAV

search and prosecute mission. We denote the time taken by Uk to travel from a point

P to another point P′ using the roundabout path with minimum radius as τUk (P,P′).

This can be calculated using Eq. (4.14) or (4.15). Let τTj (P,P′) denote the time taken

by target Tj to travel from P to P′. We first check whether it is possible for Uk to

intercept Tj before it crosses the boundary. If τUk(GTjUk

, ITj

B) > τTj(ITj

0 , ITj

B), then Uk

cannot intercept the target before it crosses the boundary and hence will not respond to

the coalition participation request; else, choose a point (say, midpoint) between ITj

0 and

ITj

B . Denote this point by ITjm . Now, use the bisection method given in Algorithm 5.1

to find the point at which Uk can intercept Tj at the earliest. Given this point, the path

length to that point can be found using Eq. (4.14) or (4.15) from which the ETA to

target, τTjUk

, can be calculated. Figure 5.4(b) illustrates two iterations of the bisection

method described in Algorithm 5.1.

151

12U2

U1

(a) Dubins paths to moving target

12U2

U1

(b) Roundabout paths to moving target

Figure 5.4: Dubins and roundabout paths to a moving target.

UAV

Target

ITjm

(1)ITj

(2)

B,

ITj0

(1)

ITj0

(2),

ITjB

(1)

GTjUk

1

2 ITjm

(2)

Boundary

Figure 5.5: Illustration of the iterative method to find ETA to a moving target usingroundabout path.

152

Algorithm 5.1: Algorithm to iteratively find the earliest interception point to a constantvelocity target

Input: Current position (ITj

0 ) of target Tj, the point at which it crosses the boundary

of the region of interest (ITj

B ), and the error tolerance for the iteration scheme

Output: Point of earliest interception of the target

Initialize: ITj

0

(0)← ITj

0 , ITj

B

(0)← ITj

B

ITjm

(0)←midPoint([ITj

0 ITj

B ])

for i = 1,2, . . . do

if τUk(GTjUk

, ITjm

(i−1)) < τTj(I

Tj

0

(i−1), I

Tjm

(i−1)) then

ITj

0

(i)← ITj

0

(i−1), I

Tj

B

(i)← ITjm

(i−1), I

Tjm

(i)←midPoint([ITj

0

(i−1)ITjm

(i−1)])

else

ITj

0

(i)← ITjm

(i−1), I

Tj

B

(i)← ITj

B

(i−1), I

Tjm

(i)←midPoint([ITjm

(i−1)ITj

B

(i−1)])

end if

if length([ITj

0

(i)ITj

B

(i)]) < tolerance then

return midPoint([ITj

0

(i)ITj

B

(i)])

break

end if

end for

153

5.3.3 Maneuvering targets

An accurate estimation of ETA is required for synchronization among UAVs to achieve

a simultaneous strike at the target. However, in case of maneuvering target, as achiev-

ing a simultaneous strike can be difficult, we follow a different strategy for prosecution

as explained in next section. Therefore, the role of ETA, in the case of maneuvering

targets, is only to decide the order in which potential coalition members will be re-

cruited into the coalition (stage 1 of PTCFA, cf. Algorithm 4.1), and hence is not that

critical. Hence, we compute ETA as the time a UAV takes to travel from its go-ahead

location (the location of a UAV at the end of coalition formation process) to the point

where the target was initially detected; this information is available in the coalition

participation request message sent by the coalition leader.

5.4 Prosecution of targets

In this section, we describe the prosecution procedure for stationary, constant velocity,

and maneuvering targets.

5.4.1 Stationary target

The prosecution strategy for stationary targets is the same as that in Chapter 4 – simul-

taneous strike to inflict maximum damage. A minor difference between the coalition

formation algorithm in Chapter 4 and that proposed in this chapter is that the coalition

formation algorithm, as modified in this chapter, takes into account network delays.

In Chapter 4, the coalition formation was assumed to be instantaneous and therefore

the prosecution maneuvers also start at the time of detection of the target. However,

with network delays accounted for, the prosecution maneuver starts at the go-ahead

time decided by the coalition formation leader at the time of initiating the coalition

formation process.

5.4.2 Constant velocity targets

To inflict maximum damage during prosecution, the UAVs in a coalition are required

to simultaneously arrive at the target. Let τTjUk

be the ETA to target Tj of UAV Uk.

Since the UAV velocity is assumed to be greater than the target velocity, it is clear that

the UAV Uk can intercept a target at any time greater than or equal to τTjUk

. Thus, the

coalition leader selects that ETA of that UAV in the coalition which has the highest

154

ETA to target as the coalition time τTjc . Every UAV in the coalition adjusts its path to

intercept the target at τTjc as follows.

Given τTjc , the location of the target at that time can be estimated from the target

information broadcast by the coalition leader. A UAV Uk calculates its roundabout

path length from its go-ahead location GTjUk

to the target’s estimated location at τTjc

using Eq. (4.14) or (4.15). Then, it iteratively determines the radius of turn that will

cause the UAV to intercept the target exactly at τTjc . Every UAV in the coalition doing

so will ensure the simultaneous arrival at the target and its subsequent prosecution.

5.4.3 Maneuvering targets

A simultaneous strike on a randomly maneuver target using UAVs moving at constant

speeds1 can turn out to be extremely difficult. Therefore, for the case of maneuvering

target, we relax the requirement of simultaneous strike. The UAVs in the coalition,

therefore, prosecute the target sequentially; each UAV follows its quickest path to the

target.

Since the target is maneuvering randomly, it is required to continuously track the

target. The UAV that tracks a target for other coalition members updates them on the

position of the maneuvering target periodically.

During prosecution, after each UAV in the coalition prosecutes it, the target ve-

locity reduces by a random amount and the target makes a turn either to the left or to

the right (randomly chosen) with a specified turn radius for a random duration of time.

This is done to mimic the behavior of a target after being partially hit.

To illustrate the prosection procedure for a maneuvering target, we give an example

of 2 UAVs and a target with initial conditions as shown in Fig. 5.6(a). In this example

U1 detects the target and becomes the coalition leader. We consider a few different

cases.

Case 1: U1 has sufficient resources to prosecute the target alone. In this case, U1

proceeds to prosecute the target by itself. This is shown in Fig. 5.6(b).

Case 2: U2 is the only member in the coalition formed to prosecute target. In this case,

although U1 is not part of the coalition, since the target is not within the sensor range

of the coalition member U2, the UAV U1 will track the target and periodically updates

U2 on the position of the target until the target is within the sensor range of U2. This is

depicted in Fig. 5.6(c).

1Rendezvous of multiple UAV onto a slowly maneuvering target, by each UAV varying its speedwithin allowable velocity bounds, is dealt with in Chapter 6

155

−400 0 400−300

0

400

U1

U2

T1

meters

met

ers

(a)

−400 0 400−300

0

400

U1

U2

T1

meters

met

ers

Interception point

(b)

−400 0 400−300

0

400

U1

U2

T1

meters

met

ers

Interception point

(c)

−400 0 400−300

0

400

U1

U2

T1

meters

met

ers

Interception points

(d)

Figure 5.6: Prosection strategy for maneuvering targets: various cases

Case 3: Both U1 and U2 are in the coalition formed by U1 to prosecute the target. In

this case, U1 proceed to prosecute the target. After the interception with target and

prosecuting it, U1 continues to track the partially hit target, periodically communicat-

ing with U2 to give updated location of target, until the target comes within the sensor

range of U2, after which, U1 continues its search mission and U2 proceeds to intercept

the target. This scenario is illustrated in Fig. 5.6(d).

5.5 Simulation Results

The performance of the coalition formation algorithm for UAVs with limited com-

munication ranges in a search and prosecute mission is studied via simulations. Ini-

tially, we present examples to show how the coalition formation is carried out and then

present Monte Carlo simulation results. Through Monte Carlo simulations, we study

the effects of the maximum number of allowed hops (Hmax) and the hop delay Δhop on

the performance of a search and prosecute mission.

For all the simulations presented in this section, the UAVs are assumed to have a

maximum turn rate of 20 deg/s. We consider the following behavior for a maneuvering

156

target in simulations: it moves ahead with the initial velocity for a random period

of time bounded by 5 s, and then takes a turn randomly either to left or right with a

turn radius of 100 m for a random time interval in the range of 0 to 3 s. This process

is repeated. After being hit by a UAV, the velocity of a target reduces by a random

amount and it does a turn maneuver either to left or right for a random time duration

which is less than 3 s. A UAV that is tracking a maneuvering target communicates

with the other UAVs, at every 1 s interval, to provide the information of the updated

position of the target.

5.5.1 Example scenario: stationary and constant velocity targets

We consider a scenario with four UAVs and two targets in a bounded region of 1000 m

× 1000 m as shown in Fig. 5.7. Of the two targets, one is static and the other is

moving. The target resource requirements are RT1 = (3,2,2) and RT2 = (2,1,2). The

moving target T2 has a velocity of 8 m/s with an initial heading of 0 degrees. The

resource capabilities of the UAVs are RU1 = (1,0,1), RU2 = (2,1,3), RU3 = (2,2,0),

and RU4 = (1,2,1). The UAVs have different but constant speeds given as V1 = 15 m/s,

V2 = 18 m/s, V3 = 20 m/s, and V4 = 24 m/s. Figure 5.7 shows the initial placement

of the UAVs and the targets. The dotted circles represent the sensor range of the

vehicles, and the dotted lines connecting the vehicles represents the communication

links between them. The sensor range is 150 m and the communication range is 2.5

times the sensor range, that is, 375 m. The maximum number of allowed hops for a

message is 2 and Δhop = 0.1 s, Δw = 2Δhop, and Δc = 0.3 s.

The sequence of events during the simulation are as follows. At time t = 4 s, U3

detects T1 and since it does not have enough resources, it broadcasts the proposal.

The UAV U3 sets the go-ahead time as τT1G = 5.5 s. The coalition participation request

message from U3 reaches the UAVs U2, U4 and U1 within Δhop seconds (at t = 4.1 s) as

these UAVs are within the communication range of U3. The information reaches the

neighbors after Δhop seconds (t = 4.1). The UAVs accept the request as they are going

to be within the communication range of U3 till t = 5.5 s, but wait for Δw seconds to see

if any other coalition request reaches them. Since, the network is small and there are

no more proposals, the neighboring agents reply to U1 at t = 4.3 s with their resources

and ETA to target. The coalition leader receives the replies by t = 4.4 s from U1, U2,

and U4 with their ETA as 49.82 s, 30.22 s, and 22.58 s. The coalition leader U3 includes

itself in the set of potential coalition members and determines a feasible coalition as

follows:

157

0 100 200 300 400 500 600 700 800 900 1000 11000

100

200

300

400

500

600

700

800

900

1000

meters

met

ers

T1

T2

12

3

4

Figure 5.7: Initial positions of the UAVs and the targets for the given example.

The coalition leader uses the optimal coalition formation algorithm (OCFA) in

Section 4.2.2. The algorithm, as described earlier, works in two stages. In the first stage

a minimum time coalition is formed and then a minimum size coalition is obtained.

The minimum time coalition formed, after the first stage, consists of UAVs U4, U3,

and U2 with coalition time as 30.22 s. The minimum size coalition obtained after the

second stage comprises U3, and U2. The coalition leader broadcasts the decision to the

UAVs which reaches them by t = 4.8 s. The accepted UAVs U3 and U2 adjust their

trajectories, as shown in Fig. 5.8, to prosecute the target simultaneously. Note that

although the confirmation message reaches all the UAVs in the coalition by t = 4.8 s,

they start the coalition maneuver, as per the protocol, only at the pre-determined go-

ahead time which is τT1G = 5.5 s.

As the mission progresses, at time t = 104 s, target T2 is detected by UAV U4. By

this time, the target T2 which initially set out with a heading of 0 degrees has reached

the boundary and taken an arbitrary turn back into the boundary by virtue of which it

has a different current heading. The UAV U4 sends a coalition formation request to

its immediate neighbors U1 and U2. Then, U1 re-broadcasts the message to U3 that

is within its communication range. The coalition leader U4 receives the responses

from U1 and U2 by t = 104.4 s. The coalition leader waits till t = 104.6 s for the

158

0 100 200 300 400 500 600 700 800 900 1000 11000

100

200

300

400

500

600

700

800

900

1000

meters

met

ers

T1

T2

1

2

3

4

Trajecory

Figure 5.8: Trajectories that the coalition members take to prosecute T1.

proposal from U3. However, U3 does not send a proposal due to lack of resources

which have been consumed while prosecuting target T1. Including itself also into the

set of potential coalition members, the coalition leader U4 executes Algorithm 4.3 to

form a coalition consisting of U2 and U4. These UAVs simultaneously prosecute the

target at the interception point, shown as a square in the Fig. 5.9, determined by the

coalition leader. The trajectories taken by UAVs U2 and U4 are shown in the same

figure. The complete mission is accomplished in 126.9 s.

5.5.2 Example scenario: maneuvering targets

We now give an example involving randomly maneuvering target. In this example, we

consider 4 UAVs and 2 targets. The initial positions of the UAVs and the targets are

shown in Fig. 5.10. The UAVs have initial speeds as V1 = 20, V2 = 18, V3 = 12, and

V4 = 15 meters per second, respectively. The target T1 has an initial speed of 5 m/s and

T2 has an initial speed of 10 m/s. The resource capabilities of UAVs are RU1 = (1,0,1),

RU2 =(3,0,0), RU3 =(1,2,1), and RU4 =(2,1,2). The resource requirements of targets

are as follows: RT1 = (2,3,2) and RT2 = (3,0,1).

After the start of the mission, at t = 4.6 s, the target T1 is detected by U1 and

initiates a coalition formation process over the UAV network which, at that time, is

159

0 100 200 300 400 500 600 700 800 900 1000 11000

100

200

300

400

500

600

700

800

900

1000

meters

met

ers

T2

1

2

3

4

Interception point

Trajectory

Figure 5.9: Trajectories that the coalition members take to prosecute the moving targetT2.

−500 0 500−500

0

500

U1

U2

U3

U4

T1

T2

meters

met

ers

Figure 5.10: Initial position of UAVs and targets for the given example.

160

−500 0 500−500

0

500

U1

U2

U3

U4

T1

T2

meters

met

ers

I3

I4

Figure 5.11: Prosecution of T1 by U3 and U4.

completely connected. The network parameters for this example are Hmax = 3, Δhop =

0.1 s, Δw = 0.2 s, Δc = 0.3 s, and a communication range of 500 m for all UAVs. The

coalition leader U1 sets the go-ahead time as 3HmaxΔhop +Δw +Δc = 1.4 seconds after

the target detection time, and therefore, a coalition {U3,U4} is formed at t = 6.0 s. For

coalition formation, U1 used the PTCFA described in Section 4.2.1. Since the target is

not in the sensor range of either U3 or U4, the coalition leader U1, although not in the

coalition, tracks the target and broadcast its updated position information periodically

until the target is within the sensor range of one of the UAVs in the coalition (U3 in

this case). The UAV U3 prosecutes the target at t = 23.0 s and U4 at t = 26.9 s. The

trajectories of U1, U3, U4, and T1 from the target detection time till its elimination is

shown in Fig. 5.11. The locations at which U3 and U4 intercepts T1 is marked in the

figure as I3 and I4, respectively.

At t = 27.1 s, immediately after the prosecution of T1, the target T2 is detected by

U1. This is shown in Fig. 5.12. After the prosecution of T1, the resource status of the

UAVs are as follows: RU1 = (1,0,1), RU2 = (3,0,0), RU3 = (0,0,0), RU4 = (1,0,1).

As seen in Fig. 5.12, the only UAVs that are in the communication range of U1 are

U3 and U4. However, the combined resources of all the three UAVs are not enough to

prosecute the target. Therefore, the search is continued.

Further in the mission, at t = 131.6 s, the target T2 is again detected, this time, by

161

−500 0 500−500

0

500

U1

U2

U3

U4

T2

meters

met

ers

Figure 5.12: A successful coalition for target T2 is not formed due to lack of resources.

U2 which initiates a coalition formation process. A successful coalition of {U1,U2}is formed at the end of go-ahead time of this coalition formation process, t = 133 s.

The UAVs U2 and U1 proceed to intercept the target T2 at t = 148.3 s and t = 150.5 s,

respectively. The trajectories of U1, U2, and T2, from the time of detection of T2 until

its complete prosecution is shown in Fig. 5.13.

5.5.3 Effect of varying number of UAVs and targets

We study the performance of the coalition formation algorithm with the increase in

number of UAVs used for the search and prosecute mission. The performance is mea-

sured in terms of the time required for mission completion.

The parameters used for the simulation are as follows. A bounded region of 1000 m

× 1000 m is considered for simulations. The sensor range is 150 m and the communi-

cation range is 300 m. The maximum number of allowed hops for a message is 3 and

Δhop = 0.1 s, Δw = 0.2 s, and Δc = 0.3 s. UAVs have fixed but different velocities in the

range 10 to 20 m/s randomly chosen at the start of each simulation. To start with, every

target has a fixed velocity between 0 and 10 m/s. We consider the target kinematics as

in Section 5.1.2. The prosecution of targets is as per Section 5.4. Also, if the target is

near the boundary, the UAV that detected it will track the target until it comes within

the region of interest and then a coalition request is made. If a UAV, which is part

162

−500 0 500−500

0

500

U1

U2

U3

U4

T2

meters

met

ers

Figure 5.13: Prosecution of T2 by U1 and U2.

of a coalition, detects a target on its way to prosecution of some other target, then it

broadcasts this information to nearby UAVs. Those UAVs that receive this message

moves to the region where the target was detected. This leads to a directed search that

will aid a quicker prosecution of targets which otherwise would have to be detected

again.

The simulation is done for a maximum simulation time of 1500 s. If the mission is

not complete (that is, all the targets are not prosecuted) within that time, then the mis-

sion time is deemed to be 1500 s. The targets are assumed to have 3 type of resources.

The quantity of each type is a random integer between 0 and 3. The UAVs also have

similar resource distribution. However, if the total UAV resources are not enough to

prosecute the target, then resources are added to UAVs so that the mission is feasible.

The above setting is used for all the remaining simulations in this chapter.

Figure 5.14 gives the total mission time, that is, the time required to find and

prosecute all the targets, as a function of the number of UAVs used in the mission.

Cases where the number of target are 5, 10, and 15 are considered. The results reported

are averages over 100 Monte Carlo runs with different initial positions and orientations

of UAVs and targets. As seen in the figure, for a fixed number of targets, the time to

prosecute all the targets decreases as the number of UAVs deployed increases.

163

0 5 10 15 20 25 30 35 4050

100

150

200

250

300

350

400

450

500

550

Number of UAVs

Mis

sion

Tim

e (s

)

15 Targets10 Targets 5 Targets

Figure 5.14: Variation of mission time with increase in number of UAVs for variousnumber of targets.

5.5.4 Effect of varying communication range

We study the effect of communication ranges of UAVs on the completion time of the

search and prosecute mission. Simulation settings are the same as described above

except for the communication range which is varied. The results obtained are shown

in Fig. 5.15. As expected, the mission performance increases with increase in com-

munication ranges of UAVs. This is because with more communication range it is

possible for the coalition leader to interact with more number of UAVs leading to a

higher chance of a successful coalition being formed. At very high values of commu-

nication range, a UAV will be able to communicate with all other UAVs without the

need for a relay agent. In which case, the problem becomes equivalent to that with

global communication as described in Chapter 4.

5.5.5 Effect of hop delay and max-hops

The goal of this experiment is to study the effects of communication delays and num-

ber of maximum allowed hops on the mission performance. The simulation setting is

the same as described before (Section 5.5.3).

Figure 5.16 shows the variation of average mission time (averaged over 100 runs)

taken by 10 and 20 UAVs to prosecute 10 targets with respect to Hmax. We study the

164

Figure 5.15: Variation of mission time with increase in communication range of UAVsfor a mission with 10 UAVs and 10 targets.

Figure 5.16: Variation of mission time with increase in maximum hops allowed for 10and 20 UAVs prosecuting 10 targets.

165

1 2 3 4 5

200

400

600

800

1000

1200

1400

Maximum hops allowed (Hmax

)

Mis

sion

Tim

e (s

)

Δhop

= 0.1

Δhop

= 0.5

Δhop

= 1

Figure 5.17: Effect of hop delay Δhop and maximum number of allowed hops Hmax onmission performance (different curves correspond to Δhop = 0.1, 0.5, and 1 s).

effect of increase in Hmax for a given Δhop = 0.1 s. Consider the change in mission time

for Hmax varying from 1 to 5 as shown in the Fig. 5.16. With the increase in Hmax, the

performance improves, that is, the mission time decreases. This is because, as the net-

work depth is more, the coalition leader can get more potential coalition members than

being restricted to only the immediate neighbors as in the case of a low value of Hmax,

say, Hmax = 1, and hence will be able to make feasible coalitions which otherwise

may not have been possible. However, further increasing Hmax will result in a slight

degradation of performance which occurs because the communication protocol that

we employ requires a sub-network that is invariant throughout the coalition formation

period. As it is difficult to find a wider network (higher Hmax) that stays invariant for a

specified duration of time, the mission time increases with Hmax, and even more so in

the case where Δhop is high (For the cases presented in Fig. 5.16, Δhop = 0.1 s). Thus,

the degradation in mission performance with increase in Hmax will be more prominent

for high Δhop. This effect is clearly seen in Fig. 5.17 where the combined effect of

Hmax and Δhop is captured. The figure shows the change in mission time for a search

and prosecute task with 10 UAVs and 10 targets with respect to maximum number

of allowed hops for different values of communication delay. Note the degradation

of mission performance for the cases where Δhop = 0.5 and 1 s when the number of

maximum allowed hops is more than 3.

166

Figure 5.18: Variation of mission time with increase in communication delay for 10and 20 UAVs prosecuting 10 targets.

Figure 5.18 shows the effect of communication delay on the mission completion

time for search and prosecute missions with 10 and 20 UAVs and 10 targets. As seen

from the figure, the mission time increases with increase in communication delay. This

is because an increase in communication delay will cause an increase in individual

coalition formation period. Since our communication protocol depends on finding a

network that has an invariant topology throughout the coalition formation process, it

becomes harder to find such a network. This results in the inability of a coalition

leader to find potential members to form feasible coalitions leading to higher mission

completion times.

From these studies, we conclude that the hop delay in network and the selection of

Hmax are important for the performance of the mission in terms of mission time. Also,

note that all the simulations had UAVs with different speed which shows the efficacy

of the algorithm for use with heterogeneous UAVs.


In this chapter, we addressed the problem of search and prosecution when the UAVs

have limited communication ranges. Toward this, we proposed a communication pro-

tocol, based on concepts from internet protocol, to form coalitions over a network

with time varying topology. Apart from stationary targets, we considered the cases of

search and prosecution of constant velocity and maneuvering targets. We presented a

167

method to compute ETA to a moving target in an unambiguous way; this was important

for the coalition formation algorithm. Through simulations, we showed that the pro-

posed methods suffices to form successful coalitions to prosecute targets – stationary,

constant velocity, and maneuvering – even when UAVs have limited communication

ranges. We showed that the mission performance, in this case, is dependent on the

network parameters like communication range, hop delay, and the maximum number

of hops a message is allowed to make in the network.

Until now, we have dealt with the problems of collision avoidance and coalition

formation of multiple UAVs separately. However, multiple UAV missions require these

algorithms to exist together or co-exist with other algorithms that are required for

successful mission completion. In the next chapter, we consider a few multiple UAV

missions that require collision avoidance and/or coalition formation and demonstrate

that the algorithms that we developed in this thesis can be seamlessly integrated to

work together with other algorithms for the overall success of the mission.

168

6 Collision avoidance and coalition formation inmultiple UAV missions

This chapter considers multiple UAV missions that require the applications of colli-sion avoidance and coalition formation algorithms developed in this thesis. First, theproblem of rendezvous of multiple UAVs doing collision avoidance maneuvers onthe way to the point of rendezvous is considered. To achieve rendezvous, the UAVsuse a consensus algorithm on their respective expected times of arrival at rendezvouspoint to control the velocity within some given bounds or a ‘wandering maneuver’.The efficacy of the algorithm is shown through simulations. Also addressed is themultiple UAV search and prosecute mission with collision avoidance. This missionrequires both the collision avoidance as well as the coalition formation algorithms tobe used. Simulations show that these algorithms can be used as independent guid-ance modules with only a small interference between them.

�n this chapter, we address certain UAV missions that require the use of col-

lision avoidance and coalition formation algorithms developed in the pre-

vious chapters. First, we consider the problem of rendezvous of multiple

UAVs. Such a rendezvous problem was encountered in Chapter 4 where the UAVs

in a coalition are required to rendezvous at the target to inflict maximum damage.

However, in that case, the issue of collision avoidance was neglected. On the way

to the rendezvous point, the UAVs in the coalition may encounter conflicts in trajec-

tories with other UAVs. These conflicts need to be avoided via appropriate collision

avoidance maneuvers. To achieve a deconfliction, the UAVs can employ the collision

avoidance algorithm developed in Chapter 2 (Algorithm 2.3). In this chapter, we pro-

pose a consensus based algorithm for a group of UAVs to rendezvous in the presence

of collision avoidance maneuvers.

Also addressed in this chapter is the multiple UAV search and prosecute mission

in the presence of collision avoidance. Chapters 4 and 5 made the assumption that

the UAVs are flying at altitudes separated by a safe distance which made collision

avoidance unnecessary. In this chapter, we relax this assumption and consider coali-

tion formation under collision avoidance. Since the coalition formation and collision

avoidance algorithms developed are higher level algorithms (that is, they are outer loop

guidance algorithms rather than inner loop control algorithms), they can be plugged in

as separate guidance modules with their priorities specified appropriately. In our work,

169

we assume that collision avoidance has higher priority than coalition formation.

This chapter is organized as follows. In Section 6.1 an algorithm based on consen-

sus is proposed for rendezvous under collision avoidance. We present the simulation

results of a group of UAVs rendezvousing in the presence of collision avoidance ma-

neuvers in Section 6.2. Section 6.3 addresses the problem of coalition formation under

collision avoidance. Section 6.4 concludes the chapter with a summary.

6.1 Rendezvous via consensus without collision avoidance

In Chapter 4, the rendezvous was taken care of by assuming that the UAVs will adjust

their trajectories to reach a predetermined location in space at a predetermined time.

However, because of collision avoidance maneuvers, it can so happen that a UAV

deviated too much from the nominal trajectory during deconfliction that it cannot reach

the predetermined location at the specified time even if it travels at the maximum

speed. In such cases, the UAVs in the coalition need to communicate with each other

and arrive at a new rendezvous time. In this section, we propose a way to handle this.

We assume that the UAVs rendezvousing to the same point (that is, UAVs in the same

coalition) do not try to avoid each other. Since rendezvousing to a point will cause

a collision, it is assumed that the UAVs will make a split maneuver when they reach

within a specified radius of the rendezvous point.

6.1.1 The consensus protocol

We propose that rendezvous be achieved by UAVs through a consensus on their Esti-

mated Time of Arrival (ETA) at the rendezvous point. The estimated arrival time for

Ui is denoted as ETAi. For a UAV Ui, we have the following kinematics and control.

xi = Vi cosψi (6.1)

yi = Vi sinψi (6.2)

ψi = ai (6.3)

Vi = fi(ETANi) (6.4)

where Vi is its velocity, ai is the angular acceleration and belongs to {−Vi/Rmin, 0,

+Vi/Rmin} (that is, UAV either turn left, goes straight, or turn right with minimum

turn radius Rmin), and ETANi = {ETA j : j ∈Ni} (set of ETAs of all the neighbors of

170

i). The set of functions fi, i = 1, . . . ,N, is the consensus protocol. In particular, we

consider the average consensus protocol given as

fi(ETANi) = ETAi− 1|Ni| ∑

j∈Ni

ETA j (6.5)

Also, we impose bounds on UAV velocity, that is,

Vmin ≤Vi ≤Vmax i = 1, . . . ,N (6.6)

If the communication is global, then Ni = {1, . . . ,N}. The next subsection discusses

the determination of ETA to target.

6.1.2 Determining ETA to target

It is assumed that all UAVs turn with a minimum radius denoted by Rmin. To find the

ETA to target (rendezvous point) using Dubins path, we need to consider two cases as

shown in Fig. 6.1. We consider only those cases where the UAVs start from locations

sufficiently far away from the target. In case (a), the target is to the left of the UAV

and in case (b), the target is on the right hand side of the UAV. In both cases, we

draw a circle tangent to the velocity vector of the UAV toward the target. Due to the

assumption that the target is far enough away from the UAV, this circle will not encircle

the target. For case (a), the center of this circle is given as

xc = xi−Rmin sinψi

yc = yi +Rmin cosψi (6.7)

and for case (b), it is

xc = xi +Rmin sinψi

yc = yi−Rmin cosψi (6.8)

We find different angles (θ1,θ2, and θ4), as shown in Fig. 6.1 for case (a) as (refer

Fig. 6.1(a))

θ1 = arctan

(yT − yc

xT − xc

)

θ2 = arccos

(Rmin√

(xT − xc)2 +(yT − yc)2

)(6.9)

θ3 =π2−ψi

171

q1

q2

q3

(xT,yT)

(xi,yi)

(xc,yc)

ji

(a)

q1

q2

q3(xT,yT)

(xi,yi)

(xc,yc)

ji

(b)

Figure 6.1: Calculating the Dubins distance to a target.

where, (xT ,yT ) is the target location and for case (b) as (refer Fig. 6.1(b))

θ1 = arctan

(yT − yc

xT − xc

)

θ2 = arccos

(Rmin√

(xT − xc)2 +(yT − yc)2

)(6.10)

θ3 =π2

+ψi

Now, the ETA to target can be obtained, for case (a), as

ETAi =√

(xT − xc)2 +(yT − yc)2−R2min +Rmin (θ3 +θ1−θ2)mod2π (6.11)

and for case (b) as

ETAi =√

(xT − xc)2 +(yT − yc)2−R2min +Rmin (θ3− (θ1 +θ2))mod2π (6.12)

where, θ mod2π is defined as the addition of θ with an integral multiple of 2π such

that θ ∈ [0,2π).

6.1.3 Handling velocity bounds

A UAV Ui chooses an angular acceleration ai which takes it toward the target (this can

be found using Algorithm 2.2). At the same time, it updates its velocity using Eq. (6.4)

according to the consensus protocol in Eq. (6.5). To take into account the upper and

lower velocity bounds as in Eq. (6.6), we have the following behavioral rules.

Rule 1 If a UAV’s velocity hits the upper bound, it stays with a velocity of Vmax until Vi

takes a value less than Vmax as updated by Eq. (6.4).

Rule 2 If a UAV’s velocity hits the lower bound, it stays with Vmin until Vi takes a value

greater than Vmin as updated by Eq. (6.4) and at the same time, maneuver away

from the target in an attempt to increase its ETA.

172

−200 −100 0 100 200

−300

−200

−100

0

100

200

300

400

500

600

UAV Trajectories

0 10 20 300

10

20

30

40

time (s)E

TA

(s)

0 10 20 3012

14

16

18

U1 U2

time (s)

velo

city

(m

/s)

Figure 6.2: Rendezvous of two UAVs using velocity control.

The wandering maneuver for a UAV, required by Rule 2, is obtained by applying an

acceleration negative to that which takes a UAV to the target.

It is shown through simulations that the velocity update using a function as in

Eq. (6.5), along with the above simple rules suffices for multiple UAV rendezvous to

occur.

6.1.4 Examples

For the simulations in this chapter, all UAVs use a radius of turn of 100 m. Figure 6.2

shows a case of two UAVs where consensus is reached on their ETAs by adjusting

the velocities of the individual UAVs. The figure shows the UAV trajectories and the

variation of their velocities and ETAs with time. In this case, as seen in Fig. 6.2, none

of the UAVs hit the velocity bounds.

Figure 6.3 shows a case of rendezvous of two UAVs starting at arbitrary locations

with arbitrary orientations, where UAVs U1 hits the lower velocity bound and does a

‘wandering maneuver’ to increase its ETA while U2 hits the upper velocity bound. A

wandering maneuver, as described earlier, consists of a turn in the direction opposite

to what would have taken it to target. Figure 6.3 clearly shows the increase in ETA of

U1 with the wandering maneuver.

173

−200 0 200 400 600−600

−400

−200

0

200

400

600

800

1000

UAV Trajectories

0 20 40 600

20

40

60

80

time (s)

ET

A (

s)

0 20 40 605

10

15

20

25U1

U2

time (s)

velo

city

(m

/s)

Figure 6.3: Rendezvous of two UAVs, U1 and U2, using velocity control and a wan-dering maneuver by U1.

To demonstrate the efficacy of the proposed rendezvous algorithm, we show the

rendezvous of 10 UAVs starting at arbitrary locations with arbitrary orientations in

Fig. 6.4. It is also possible for a group of UAVs to rendezvous on to a slowly mov-

ing/maneuvering target by considering the rendezvous location as the current location

of the target. Example of such a rendezvous is shown in Fig. 6.5.

6.1.5 Discussion

Unlike the traditional consensus, where all the agents converge to a constant consensus

value, in the present case, the consensus value – the ETA to target – changes with time

as UAVs are in motion.

In the simulations till now, we used the average consensus protocol. In this case,

the rate of change of velocity is determined by Eq.(6.5). For cases where number

of UAVs that need to rendezvous are high, such a consensus protocol may result in

slow convergence. This will be more prominent in cases where most of the UAVs in

the group have their ETA values close to each other (as a result, close to the group

average), while the ETAs of a few UAVs are significantly different from the group

average. Such a situation may also arise when the UAVs reach closer to the rendezvous

point: a few UAVs ‘lagging’ behind has already reached their upper speed limit, while

174

−1000 −500 0 500 1000−1000

−800

−600

−400

−200

0

200

400

600

800

1000

0 10 20 30 40 500

20

40

60

80

0 10 20 30 40 505

10

15

20

25

UAV Trajectories

time (s)

velo

city

(m

/s)

time (s)

ET

A (

s)

Figure 6.4: Rendezvous of 10 UAVs using velocity control within bounds and wander-ing maneuvers.

−200 0 200

−500

−400

−300

−200

−100

0

100

200

300

400

UAV trajectories

meters

met

ers0 10 20 30

0

10

20

30

40

time (s)

ET

A (

s)

0 10 20 305

10

15

20

25

time (s)

spee

d (m

/s)

target trajectory

Figure 6.5: Rendezvous of 3 UAVs onto a slowly maneuvering target.

175

−500 0 500

−1500

−1000

−500

0

500

1000

1500

UAV trajectories

0 20 40 600

20

40

60

80

100

time

ET

A (

s)

0 20 40 605

10

15

20

25

time (s)

velo

city

(m

/s)

slow convergence

Figure 6.6: Rendezvous of 20 UAVs using average consensus.

most of the other UAVs have ETAs that are too close to the group average to make

necessary maneuvers to ‘wait’ for the few lagging behind. This slow convergence

behavior (note that as time progresses, the ETAs do not quite merge into a single line)

is seen in Fig. 6.6 which shows the rendezvous of 20 UAVs using the average consensus

protocol.

The convergence can be made faster by use of a different consensus protocol. For

example, consider the following function for velocity update

fi(ETANi) = ETAi− min j∈Ni(ETA j)+max j∈Ni(ETA j)2

(6.13)

In this protocol, a ‘weighted average’ of ETAs, instead of the mean of ETAs, is used. In

particular, the protocol uses the mean of the maximum and minimum of ETA values of

all the UAVs. This helps to mitigate the problem of slow convergence described above

to an extent. For the same initial conditions as in the case depicted in Fig 6.7, using

Eq.(6.13) instead of Eq.(6.5) for velocity update results in a faster convergence of ETA

as seen from Figure 6.7. Note that the convergence of ETA values of different UAVs

into a single line is better in this case when compared to the convergence in Fig. 6.6.

The lesser ‘spread’ or a better convergence implies a more accurate rendezvous.

A rendezvous is possible even when only local communication is possible. That

is, the UAVs have limited communication ranges and therefore can communicate with

176

−500 0 500

−1500

−1000

−500

0

500

1000

1500

UAV trajectories

0 20 40 600

20

40

60

80

100

timeE

TA

(s)

0 20 40 605

10

15

20

25

time (s)

velo

city

(m

/s)

Figure 6.7: Rendezvous of 20 UAVs using weighted average consensus.

only those UAVs that are within its communication range. The requirement for a ren-

dezvous to occur in that case is that the underlying communication graph, formed by

the limited range communication UAVs, should eventually be connected. We con-

sider a case in which every UAV has a communication range equal to 300 m. The

rendezvous of 20 UAVs under this communication constraint is shown in Fig. 6.8.

Figure 6.9 shows the rendezvous, for same initial conditions, under global communi-

cation. For rendezvous using local communication, we note from Fig. 6.8 that initially

sub-groups that are formed by UAVs close to each other attain individual local consen-

sus, which, as time progresses coalesce into a single global consensus. From Fig. 6.8,

we can observe that at around 10 s, at least two distinct local consensus which later

merges into a global consensus.

6.2 Rendezvous with collision avoidance

Now we show that the rendezvous algorithm that we developed can be used to achieve

a rendezvous under collision avoidance.

6.2.1 Incorporating collision avoidance

The UAVs trying to converge to a point avoid other UAVs on the way. Algorithm 2.3

177

−500 0 500 1000

−1500

−1000

−500

0

500

1000

1500

UAV trajectories

meters

0 20 40 600

20

40

60

80

100

ET

A (

s)

0 20 40 605

10

15

20

25

time (s)

velo

city

(m

/s)

Figure 6.8: Rendezvous of 20 UAVs using limited communication.

−500 0 500 1000

−1500

−1000

−500

0

500

1000

1500

UAV trajectories

meters

0 20 40 600

20

40

60

80

100

ET

A (

s)

0 20 40 605

10

15

20

25

time (s)

velo

city

(m

/s)

Figure 6.9: Rendezvous of 20 UAVs using global communication.

178

−500 0 500

−1000

−500

0

500

1000

1500

U1

U2

U3 U

4

U5

UAV Trajectories

meters

met

ers0 20 40 60 80

0

20

40

60

80

100

time (s)E

TA

(s)

0 20 40 60 805

10

15

20

25

time (s)

velo

city

(m

/s)

Figure 6.10: Rendezvous of 5 UAVs with collision avoidance.

is used to effect a deconfliction. It is assumed that the other UAVs are non-cooperating

and do not do a collision avoidance maneuver which is a more stringent case. Collision

avoidance maneuvers by a UAV will cause an increase in its ETA to target. However,

the adaptation of velocity through consensus in conjunction with the wandering ma-

neuver will ensure that the UAVs still achieve rendezvous. We do not consider collision

avoidance among the UAVs that are rendezvousing (we assume that they split off at

the end of rendezvous).

6.2.2 Simulation results

Figure 6.10 shows rendezvous of 5 UAVs where three of them avoid collisions with

other UAVs on the way to the target. Our rendezvous algorithm is seen to aid a ren-

dezvous even when the involved UAVs do collision avoidance maneuvers to avoid

trajectory conflicts with other UAVs. Figure 6.11 shows that the collision avoidance

maneuvers causes changes in ETAs of the corresponding UAVs but are promptly nul-

lified by the consensus protocol to guarantee a rendezvous.

We now turn our attention to rendezvous of a group of UAVs avoiding collisions in

a high density traffic environment. Figure 6.12 gives a picture of the congested airspace

where UAVs U1, U2, U3, and U4 have to attain a rendezvous. The figure shows a sce-

nario in which 40 other UAVs always occupy the concerned airspace; these UAVs are

179

0 10 20 30 40 50 60 70 800

10

20

30

40

50

60

70

80

time

ET

A

26 28 30

34

36

38

time

ET

A

Figure 6.11: The variation of ETAs of UAVs while attaining rendezvous in presenceof collision avoidance.

assumed to be non-cooperative, that is, they do not perform collision avoidance ma-

neuvers. A particular case of rendezvous of UAVs U1, U2, U3, and U4 with and without

collision avoidance with other UAVs is shown in Figure 6.13. The solid lines are the

paths that these UAVs take to the rendezvous point while not performing any collision

avoidance maneuvers; the dashed lines are the rendezvous trajectories with maneuvers

to avoid collisions. The performance of the collision avoidance algorithm and the con-

sensus algorithm for rendezvous in such a mission for various aircraft densities are

shown in Table 6.1. In this simulation set up, 4 UAVs, starting at a random point along

each edge of the airspace as shown in Figure 6.12, have to rendezvous at the center.

As seen in the same figure, there are other UAVs originating at random points along

one of the four edges (boundaries) of the concerned airspace and flying to the opposite

edge. During a particular simulation, the number of these UAVs in the airspace is kept

constant by introducing new UAVs to replace the ones which reached their respective

destinations. Every UAV is assumed to have a speed of 15 m/s. The UAVs U1, U2,

U3, and U4 have a minimum radius of turn of 100 m. Since the environment is ran-

dom, the values reported in Table 6.1 are averages over 30 runs. We note from the

table that use of the collision avoidance algorithm RCA-2D, developed in Chapter 2,

keeps the number of near misses to very low values for the kind of congested environ-

ment considered. In Table 6.1, the efficiency represents the ratio of the time taken for

180

−1000 0 1000−1000

0

1000

meters

met

ers

U1

U2

U3

U4

Figure 6.12: Congested airspace in which a rendezvous need to be achieved.

Table 6.1: Performance of the rendezvous algorithm in congested environments (aver-aged over 30 runs)

Aircraft Density Near Misses Efficiency (%)20 0.83 83.5040 1.10 81.5360 2.63 66.84

rendezvous with collision avoidance to the time taken without collision avoidance ma-

neuvers. The efficiencies obtained are less because the collision avoidance maneuver

of one of the UAVs will cause all the other UAVs also to alter their paths so as to attain

a rendezvous leading to a large reduction in efficiency. This is in contrast to the cases

considered before in this thesis where the deviations of a few UAVs did not affect the

overall efficiency much.

6.3 Coalition formation in a search and prosecute mission with collision avoidance

In this section, we consider a multiple UAV mission that requires both the collision

avoidance and the coalition formation algorithms to co-exist for the successful com-

pletion of the mission.

181

−1000 −500 0 500 1000

−500

0

500

meters

met

ers

U1

U2

U3

U4

Figure 6.13: Rendezvous with and without collision avoidance in a dense environment.

6.3.1 Coalition formation in presence of collision avoidance

We consider multiple UAV search and prosecute missions, as described in Chapter 5,

with the requirement that the UAVs have to avoid collisions among themselves. To-

ward this, we use the collision avoidance and coalition formation algorithms developed

in previous chapters. These algorithms are higher level algorithms. That is, they form

the part of outer guidance loop independent of the inner control loop. Therefore, it is

possible to ‘plug in’ the collision avoidance and coalition formation algorithms as sep-

arate guidance modules with care taken to specify the priorities of each according to

the mission in hand. For our search and prosecute mission, we require that the collision

avoidance is given priority to coalition formation and subsequent prosecution. For ex-

ample, if a UAV on the way to target prosecution encounters a trajectory conflict with

some other UAV, then the conflict is resolved before proceeding to prosecution. This

behavior will lead to a slight degradation of the performance of the coalition formation

algorithm in terms of the mission completion times. However, as will be demonstrated

through simulations, these deviations are not large.

Figure 6.14 shows the trajectories of two UAVs and a target during a search and

prosecute mission. The solid lines show the UAV trajectories prosecuting targets when

no collision avoidance algorithm is employed; the dashed lines show the UAV trajecto-

ries when they use RCA-2D for collision avoidance among themselves. As seen from

182

−300 −200 −100 0 100 200 300−300

−200

−100

0

100

200

300

meters

met

ers

conflict point

U2

U1

T1

solid line: without collision avoidancedashed line: with collision avoidance

Figure 6.14: Trajectories of UAVs prosecuting a target (solid lines – without collisionavoidance; dashed lines – with collision avoidance).

the figure, it takes longer to prosecute the target when the UAVs have to also avoid

colliding with each other.

6.3.2 Simulation results

Simulations are carried out to study the performance of the coalition formation algo-

rithm in conjunction with the collision avoidance algorithm for multiple UAV search

and prosecute mission. The simulation setting is the same as that described in Sec-

tion 5.5.3.

We consider only the case of prosecution of 10 targets by varying the number of

UAVs. The comparison of the mission completion times with and without collision

avoidance is shown in Fig. 6.15. The values reported and averaged over 100 Monte

Carlo runs. As seen in the figure, there is a slight degradation in the mission perfor-

mance in terms of the mission completion time. However, the deviations are not very

large. Nonetheless, as seen in Fig. 6.15, even in the case where collision avoidance al-

gorithm is used, a similar scale-up of the mission completion time is obtained with the

increase in number of UAVs as in the case without collision avoidance. The average

number of near misses observed for the cases of 10, 20, and 30 UAVs are 0.1, 0.29,

and 0.98 respectively.

183

0 5 10 15 20 25 30 35 4050

100

150

200

250

300

350

400

450

500

Number of UAVs

Mis

sion

Tim

e (s

)

without collision avoidancewith collision avoidance

Figure 6.15: Comparison of average mission times as a function of number of UAVsfor a search and prosecute task for 10 targets with and without collision avoidance.

The above result point to the fact that the collision avoidance and the coalition

formation algorithms can be integrated easily into UAV missions with very less inter-

ference between the two.


In this chapter, we developed an algorithm for rendezvous of multiple UAVs in the

presence of collision avoidance maneuvers. Toward this, we used the principles of

consensus and showed that achieving consensus on ETA (which is time varying) will

enable a rendezvous. Simulations show that using consensus protocol on ETA will

guarantee rendezvous. Next, we considered the multiple UAV search and prosecute

mission with the additional requirement that the UAVs should avoid collisions among

themselves. Although the missions took slightly longer with the collision avoidance

algorithm activated, the reduction of mission times with increase in number of UAVs

used is achieved in this case also. This points to the fact that the collision avoidance

and the coalition formation algorithms can be used in conjunction in multiple UAV

missions that require both the algorithms.

184

7 Conclusions

This chapter serves as a summary of the thesis and discusses the important resultsand the conclusions arrived at in this thesis. Some directions for future work, arisingfrom the research presented in this thesis, are also provided.

�he inevitable use of UAVs for dull, dirty, and dangerous missions lead to

the use of multiple UAVs, and subsequently to the multiple UAV opera-

tional problems like coordination and cooperation among UAVs. This the-

sis addressed the problems of collision avoidance and coalition formation of UAVs in

high density traffic environments which has applications in many multiple UAV mis-

sions. This chapter concludes the thesis with a summary, highlighting novelties and

directions for further research opened up by the work presented in this thesis.

7.1 Collision Avoidance

One of the main problems that this thesis addressed was that of collision avoidance

among UAVs in high density traffic environments. We advocated the following: When

a UAV faces multiple conflicts, it chooses the most threatening conflict and executes an

appropriate conflict resolution maneuver. We defined the most threatful conflict as one

that has the least time to go for a near miss to occur, and the appropriate conflict reso-

lution maneuver as the one that increases the LOS rate between the concerned UAVs.

Through simulations, we showed that this local strategy gives good performance in

terms of low number of near misses and higher efficiency. The algorithm is simple

and easy to implement. This algorithm was shown to be robust to noise in position

and velocity measurements. The algorithm, as it is reactive, will work even if one of

the UAVs in each pairwise conflict does not perform avoidance maneuver. This al-

gorithm was then implemented in six degree of freedom UAV model augmented with

controllers designed to execute the guidance commands. Simulations using realistic

UAV models showed good performance which is a certificate of both the effectiveness

and ease of implementation of the proposed algorithm. We also developed algorithms

to handle three dimensional conflicts. Toward this, we introduced the concept of a

185

collision plane and proposed a conflict resolution maneuver that consists of either (a)

pulling out of the collision plane, or (b) doing a turn in the collision plane; the second

being a generalization of the two dimensional collision avoidance algorithm developed

earlier to handle three dimensional conflicts.

The two and three dimensional collision avoidance algorithms developed in this

thesis were implemented, in simulation, on 6 DoF realistic UAV models. Toward this,

we designed PI-controllers using successive loop closure technique which when used

in conjunction with the collision avoidance guidance algorithms provided the 6 DoF

UAV models with the required acceleration so as to perform well in high density traffic

simulation scenarios. Nonetheless, the response of the 6 DoF UAV model augmented

with PI control loops was a bit sluggish. Also, the PI-controllers required the infor-

mation of some of the aircraft states as feedback. In this thesis, we assumed that the

required states are available, free of noise. However, in the case of real-world UAVs,

owing to the use of low cost sensors, some states are not measured and hence need to

be estimated, and the measured states are noisy. An estimator should, in principle, take

this into account. However, the main focus of this thesis was to develop conceptually

simple and effective guidance algorithms and demonstrate their ease of implementa-

tion. We, therefore, identify the development of sophisticated controllers/estimators –

to be used in high density traffic scenarios – as an area of future work. Also, neither

the controllers nor the guidance algorithms developed in this thesis take into account

situations where considerable wind is present. Although the presence of wind will not

affect the collision avoidance performance, it is critical for tracking shortest paths to

destinations.

We demonstrated the efficacy of the developed collision avoidance algorithms

through extensive simulations. Two important concepts for any collision avoidance

algorithm are stability and robustness. Broadly, stability of a collision avoidance algo-

rithm refers to its breakdown via ‘domino effect’ and/or unbounded deviations from

nominal trajectories (Mao et al., 2000), when the traffic density increases beyond cer-

tain value; robustness can be looked upon as the smooth degradation of the perfor-

mance of the algorithm with increase in uncertainty of the position, velocity, et cetera

and other disturbances. It is desirable to have theoretical results guaranteeing the sta-

bility and robustness of conflict resolution algorithms. However, as Hoekstra et al.

(2000) observes, this can be extremely difficult. They note that even simple rules, like

cellular automata, leads to disasters and any conflict resolution algorithm is more com-

plicated than cellular automata. “Consequently,” they argue, it may be difficult or even

“impossible to guarantee stability” and robustness of collision avoidance algorithms

186

and they are at best “only be studied using simulations.”

There is no effective measure to quantify the stability and robustness of decen-

tralized collision avoidance algorithms. It is possible to envisage a situation where a

decentralized deconflcition algorithm shows good performance, in terms of number

of near misses, till a specific density and then the performance degrades drastically

leading to a phase transition like behavior. However, there exist no measures either

to quantify this phenomenon or to predict the critical density at which degradation

starts. The nonexistence of such measures is an indication of the complexity of the

problem of quantifying the stability of decentralized collision avoidance algorithms.

The measures of stability defined in the literature favors centralized algorithms thus

leading to wrong conclusions about the performance of decentralized algorithms. For

example, Krozel et al. (2001) defines a stability parameter in terms of the number of

detected conflicts. They argue that if a deconfliction algorithm leads to more number

of detected conflicts, it is unstable. However, it is straightforward to see that the use

of any decentralized algorithm, that operates with only local information, will invari-

ably lead to more detected conflicts than a centralized algorithm, especially in high

density traffic situations; leave aside the fact that a centralized algorithm will be com-

putationally too expensive to handle such a dense environment. It is not the number of

conflict detections, rather the eventual resolution of them, without much compromise

in achieved efficiency, that is the true indicator of stability of an algorithm. Thus, the

measures quantifying phase transition like behaviors, as described before, will better

represent the stability of decentralized algorithms. However, such measures are yet

to emerge. Development of meaningful and useful performance measures to analyze

collision avoidance algorithms designed to handle high density traffic situations will

constitute good future work in this direction.

7.2 Coalition Formation

Another problem that this thesis addressed is the problem of coalition formation among

multiple UAVs. Although, in particular, we considered the coalition formation for a

search and prosecute task, our abstract approach to the problem makes it applicable

to any general task allocation problem. First, we proved that the minimum time mini-

mum member coalition formation problem that we address is NP-hard. Then we gave

a two stage polynomial time algorithm to solve the problem, giving a sub-optimal so-

lution. This algorithm scales well for use in high density traffic situations. This was

187

shown through simulations. Our local strategy for coalition formation gave perfor-

mance comparable to that of the global optimization problem, where the target loca-

tions are known a priori, obtained using Particle Swarm Optimization technique; while

the computational advantage of the proposed two stage algorithm is huge.

The coalition formation over a time varying network was addressed next. Such

a situation arises when the UAVs have limited communication ranges and motion of

UAVs, searching for targets, will cause the formation of a communication network that

is dynamic. Our strategy of tackling the problem of coalition formation over a time

varying network was to find a sub-network that is invariant over the coalition forma-

tion period. We presented communication protocols and showed through simulations

that our approach worked well. Dependence of the performance of the proposed algo-

rithm on mission parameters and network parameters were studied extensively. Studies

showed that the mission performance, in terms of mission completion time, depends

heavily on network parameters like delay. Our approach – finding a sub-network in-

variant over the coalition formation period – was successful as our coalition formation

algorithm is fast and thus the coalition formation time is much lesser compared to the

timescale at which the network topology changes because of the UAVs moving around

in search of targets. Such an approach will definitely fail if the network changes at

a much faster rate. Algorithms for efficiently forming coalitions over such networks

can, possibly, be designed by drawing ideas from communication protocols used in

wireless mobile ad hoc networks.

Managing the available bandwidth effectively becomes important when one is

dealing with groups of hundreds and thousands UAVs. The coalition formation algo-

rithms we proposed, although requires very low communication between the coalition

leader and the potential coalition members for the coalition formation process, the use

of broadcast and flooding mechanisms may cause a bandwidth crunch in very large

groups of UAVs caused by the presence of a large number of communicating relay

nodes in between the coalition leader and the potential coalition members. The band-

width crunch can, to an extent, be mitigated, at the cost of the mission completion

time, by reducing communication ranges and the number of hops a message is allowed

to make. There is scope for design of better communication protocols in these cases.

In this thesis, we considered a sequential prosection of a maneuvering target. An

interesting extension of this can be the simultaneous prosecution of maneuvering tar-

gets, for which, the technique of rendezvous via consensus – presented in Chapter 6

of the thesis – can be used. Also, the search and prosecution of ground targets in the

presence of wind will constitute a good extension of the present work.

188

7.3 Collision avoidance and coalition formation in multiple UAV missions

There are several multiple UAV missions that requires collision avoidance and coali-

tion formation among UAVs. Collision avoidance is a mandatory requirement in all

multiple UAV missions. We considered one such mission – multiple UAV rendezvous

under collision avoidance. The UAVs used a consensus on their expected time of

arrival at the rendezvous point to make simultaneous arrival possible, even in the pres-

ence of collision avoidance maneuvers performed by some of the UAVs en route.

The proposed rendezvous algorithm was then modified for tracking a target – sta-

tionary or maneuvering – from a standoff distance. The algorithm in its current form

does not allow controlling of the spacing between the UAVs tracking the target. In-

corporation of this feature into the consensus based standoff tracking is a potential

extension of the current work. Rendezvous and target tracking using consensus is sim-

ple and effective, as consensus based algorithms are naturally suited for multi-UAV

coordination problems (Ren & Beard, 2008), and there remains a lot to be explored in

this direction.

The rendezvous using consensus depended on the velocity control to achieve ren-

dezvous. This was possible because the collision avoidance algorithm used only turn

maneuvers for deconflcition leaving the freedom to vary velocities within bounds to

other algorithms that may have to work in conjunction, and may require this flexibility,

to make the mission successful. If the collision avoidance algorithm were to be com-

plex, using all available degrees of freedom, not much authority will be left for other

applications. This feature of leaving enough room for other applications is the charac-

teristic of a simple algorithm and is present in the algorithms developed in this thesis,

which is more explicitly seen in the mission that we considered – coalition formation

for search and prosecution in presence of collision avoidance.

As the collision avoidance and coalition formation algorithms developed in this

thesis are high level algorithms, they can be plugged in independently to the set of

functions of a UAV with priorities defined appropriately. For example, in a search and

prosecute mission with collision avoidance, the task of collision avoidance has higher

priority than the task of coalition formation, barring pathological situations. Since

these algorithms are simple, as mentioned earlier, the interaction and interference be-

tween these modules are minimal as depicted by the simulations presented.

For the search and prosecute mission for maneuvering targets, we adopted a se-

quential prosecution procedure. Simultaneous prosecution of maneuvering targets,

189

when the UAVs in the coalition also have to perform collision avoidance maneuvers,

is a good problem that finds practical application, and can be addressed using a slight

modification of our rendezvous via consensus algorithm.

7.4 Concluding remarks

Processes where locally rational or optimal interactions leading to global equilibrium,

optimality, and globally desirable behaviors are ubiquitous in nature. Flocking in birds

(Cucker & Smale, 2007) is an example. This characteristic emerges even out of some

theories that do not explicitly take into account such considerations. An example is

the best response mechanism in game theory where “each player (is) performing only

simple ‘locally rational’ actions and yet, mysteriously, the system would reach a global

equilibrium” (Nisan et al., 2011). The algorithms proposed in this thesis – collision

avoidance and coalition formation algorithms – have the characteristic of being ‘local’,

both in space and time; multiple conflicts were handled by resolving the most immi-

nent conflict, and coalitions were formed as and when the targets were detected and

communicating with only the neighbors. The good ‘global’ performance, in terms of

less number of near misses with high efficiency and small mission completion times, of

these algorithms implicitly brings out this characteristic or property – ‘locally rational

interactions leading to globally acceptable behavior’ – present in the coordination and

cooperation among UAVs in large groups. The knowledge of this characteristic can be

exploited to design useful multiple UAV mission algorithms which are conceptually

simple, yet effective and scalable.

190

Bibliography

Alagar, S., Venkatesan, S., & Cleveland, J. (1995). Reliable broadcast in mobile wireless networks.

In Proceedings of IEEE Military Communications Conference, San Diego, USA, pp. 236–

240.

Albaker, B. M., & Rahim, N. A. (2009a). Straight projection conflict detection and coopera-

tive avoidance for autonomous unmanned aircraft systems. In Proceedings of the 4th IEEE

Conference on Industrial Electronics and Applications, Kuala Lumpur, Malaysia, pp. 1965–

1969.

Albaker, B. M., & Rahim, N. A. (2009b). A survey of collision avoidance approaches for un-

manned aerial vehicles. In Proceedings of the International Conference for Technical Post-

graduates, Kuala Lumpur, Malaysia.

Alighanbari, M., & How, J. (2006). Robust decentralized task assignment for cooperative UAVs.

In AIAA Guidance, Navigation, and Control Conference and Exhibit, Keystone, USA. Paper

No. AIAA-2006-6454.

Archibald, J. K., Hill, J. C., Jepsen, N. A., Strirling, W. C., & Frost, R. L. (2008). A satisficing ap-

proach to aircraft conflict resolution. IEEE Transactions on System, Man, and Cybernetics,

Part C: Applications and Reviews, 38, pp. 510–521.

Arslan, G., Marden, J. R., & Shamma, J. S. (2007). Autonomous vehicle-target assignment :

a game-theoretical formulation. Journal of Dynamic Systems, Measurement, and Control,

129, pp. 584–596.

Banda, S. S. (2002). Future directions in control of unmanned aerial vehicles. In Proceedings of

AFOSR Workshop on Future Directions in Control, Arlington, USA.

Bicchi, A., & Pallottino, L. (2000). An optimal cooperative conflict resolution for air traffic

management systems. IEEE Transactions on Intelligent Transportation Systems, 1, pp. 221–

232.

Bilimoria, K. D. (2000). A geometric optimization approach to aircraft conflict resolution. In

AIAA Guidance, Navigation, and Control Conference and Exhibit, Denver, USA. Paper

No. AIAA-2000-4265.

191

Campbell, A., Wu, A. S., & Shumaker, R. (2002). Multi-agent task allocation: Learning when

to say no. In Proceedings of the 10th Annual Conference on Genetic and Evolutionary

Computation, New York, USA, pp. 201–208.

Carbone, C., Ciniglio, U., Corraro, F., & Luongo, S. (2006). A novel 3D geometric algorithm for

aircraft autonomous collision avoidance. In Proceedings of the 45th IEEE Conference on

Decision and Control, San Diego, USA.

Chakravarthy, A., & Ghose, D. (1998). Obstacle avoidance in a dynamic environment: a collision

cone approach. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and

Humans, 28, pp. 562–574.

Chen, J., & Huang, M. (2005). A broadcast engagement ack mechanism for reliable broadcast

transmission in mobile ad hoc networks. IEICE Transactions on Communications, 88, pp.

3570–3578.

Clark, M. P. (2003). Data Networks, IP and the Internet: Protocols, Design and Operation. John

Wiley & Sons Ltd., Chichester.

Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2001). Introduction to Algorithms.

MIT Press, Cambridge.

Cucker, F., & Smale, S. (2007). Emergent behavior in flocks. IEEE Transactions on Automatic

Control, 52, pp. 852–862.

Darrah, M., Nil, W., & Stolarik, B. (2006). UAV cooperative task assignments for a SEAD mis-

sion using genetic algorithms. In AIAA Guidance, Navigation, and Control Conference and

Exhibit, Keystone, Colorado. Paper No. AIAA-2006-6456.

Das, K., & Ghose, D. (2009). Positional consensus in multi-agent systems using broadcast control

mechanisms. In Proceedings of the American Control Conference, St. Louis, USA, pp.

5731–5736.

Dhananjay, N., Ghose, D., & Bhat, M. S. (2010). Performance analysis of guidance laws based

on timescale gap. IEEE Transactions on Control Systems Technology, 18, pp. 574–590.

Dowek, G., Munoz, C. A., & Carreno, V. A. (2005). Provably safe coordinated strategy for

distributed conflict resolution. In AIAA Guidance, Navigation, and Control Conference and

Exhibit, San Francisco, USA. Paper No. AIAA-2005-6047.

Dubins, L. E. (1957). On curves of minimal length with a constraint on average curvature, and with

prescribed initial and terminal positions and tangents. American Journal of Mathematics, 79,

pp. 497–516.

192

Eberhart, R. C., & Kennedy, J. (1995). A new optimizer using particle swarm theory. In Pro-

ceedings of the Symposium on Micro Machine and Human Science, Piscataway, USA, pp.

39–43.

Eby, M., & Kelly, W. E. (1999). Free flight separation assurance using distributed algorithms. In

Proceedings of the IEEE Aerospace Conference, San Francisco, USA, pp. 429–441.

Eby, M. S. (1994). A self-organizational approach for resolving air traffic conflicts. The Lincoln

Laboratory Journal, 7, pp. 239–254.

Fiorini, P., & Shiller, Z. (1998). Motion planning in dynamic environments using velocity obsta-

cles. The International Journal of Robotics Research, 17, p. 760772.

Fox, D., Burgard, W., & Thrun, S. (1997). The dynamic window approach to collision avoidance.

IEEE Robotics and Automation Magazine, 4, pp. 23–33.

Fraichard, T., & Asama, H. (2004). Inevitable collision states – a step towards safer robots?.

Advanced Robotics, 18, pp. 1001–1024.

Frazzoli, E., Pallottino, L., Scordio, V., & Bicchi, A. (2005). Decentralized coopperative conflict

resolution for mutiple nonholonomic vehicles. In AIAA Guidance, Navigation and Control

Conference and Exhibit, San Francisco, USA. Paper No. AIAA-2005-6048.

Frew, E. W., Lawrence, D. A., & Morris, S. (2008). Coordinated standoff tracking of moving

targets using Lyapunov guidance vector fields. Journal of Guidance, Control, and Dynamics,

31, pp. 290–306.

Furukawa, T., Bourgault, F., Whyte, H. F. D., & Dissanayake, G. (2005). Dynamic allocation

and control of coordinated uavs to engage multiple targets in a time-optimal manner. In

Proceedings of the IEEE Conference on Robotics and Automation, Barcelona, Spain, pp.

2353–2358.

Garey, M. R., & Johnson, D. S. (1979). Computers and Intractability: A Guide to the Theory of

NP-Completeness. W.H. Freeman & Co., New York.

George, J., Sujit, P. B., & Sousa, J. B. (2011). Search strategies on multiple uav search and destroy

missions. Journal of Intelligent and Robotic Systems, 61, pp. 355–367.

Gerkey, B., & Mataric, M. J. (2004). Formal framework for the study of task allocation in multi-

robot systems. International Journal of Robotics Research, 23, pp. 939–954.

Ghawghawe, S. N., & Ghose, D. (1996). Pure proportional navigation against time-varying target

maneuvers. IEEE Transactions on Aerospace and Electronic Systems, 32, pp. 1336–1347.

193

Gomez, M. L., & Fraichard, T. (2009). Benchmarking collision avoidance schemes for dynamic

environments. In Proceedings of the ICRA Workshop on Safe Navigation in Open and Dy-

namic Environments, Kobe, Japan.

Gu, D.-W., Kim, Y., & Postlethwaite, I. (2010). How tight is sphere-packed formation flying?.

Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engi-

neering, 224, pp. 427–435.

Guelman, M. (1971). A qualitative study of proportional navigation. IEEE Transactions on

Aerospace and Electronic Systems, 7, pp. 637–643.

Han, S.-C., & Bang, H. (2004). Proportional navigation-based optimal collision avoidance for

UAVs. In Proceedings of the 2nd International Conference on Autonomous Robots and

Agents, Palmerston North, New Zealand.

Hill, J. C., Archibald, J. K., Stirling, W. C., & Frost, R. L. (2005). A multi-agent system ar-

chitecture for distributed air traffic control. In AIAA Guidance, Navigation, and Control

Conference and Exhibit, San Francisco, USA. Paper No. AIAA-2005-6049.

Hoekstra, J., Ruigrok, R. C. J., & van Gent, R. N. H. W. (2000). Free flight in a crowded airspace?.

In 3rd USA/Europe Air Traffic Management R&D Seminar, Napoli, Italy.

Hwang, I., & Tomlin, C. (2002). Protocol-based conflict resolution for finite information horizon.

In Proccedings of the American Control Conference, Anchorage, USA, pp. 748–753.

IEEE-SA Standards Board (2007). IEEE wireless LAN medium access control (MAC) and physi-

cal layer (PHY) specifications. Industry standard, IEEE Computer Society, New York, USA.

Kamal, Y. A., Gu, D.-W., & Postlethwaite, I. (2005). Real time trajectory planning for UAVs using

MILP. In Proceedings of the 44th IEEE Conference on Decision and Control and European

Control Conference, Seville, Spain, pp. 3381–3386.

Kennedy, J., & Eberhart, R. C. (1995). Particle swarm optimization. In Proceedings of the IEEE

International Conference on Neural Networks, Piscataway, USA.

Kim, Y., Gu, D.-W., & Postlethwaite, I. (2007). Tight formation flying and sphere packing. In

Proceedings of the American Control Conference, New York, USA, pp. 1085–1090.

Kingston, D. B., & Schumacher, C. J. (2005). Time-dependent cooperative assignment. In Pro-

ceedings of the American Control Conference, Portland, USA, pp. 4084–4089.

194

Krozel, J., Peters, M., Bilimoria, K. D., Lee, C., & Mitchell, J. S. B. (2001). System perfor-

mance characteristics of centralized and decentralized air traffic separation strategies. In

Proceedings of the 4th USA/Europe Air Traffic Management R&D Seminar, Santa Fe, USA.

Kuchar, J. K., & Yang, L. C. (2000). A review of conflict detection and resolution modeling

methods. IEEE Transactions on Intelligent Transportation Systems, 1, pp. 179–189.

Lacher, A. R., Maroney, D. R., & Zeitlin, A. D. (2007). Unmanned aircraft collision avoidance:

Technology assessment and evaluation methods. In Proceedings of the 7th USA/Europe Air

Traffic Management Research and Development Seminar, Barcelona, Spain.

Lalish, E. (2009). Distributed reactive collision avoidance. Ph.D. thesis, University of Washing-

ton, Seattle, USA.

Lalish, E., Morgansen, A., & Tsukamaki, T. (2008). Decentralized reactive collision avoidance

for multiple unicycle-type vehicles. In Proccedings of the American Control Conference,

Seattle, USA, pp. 5055–5061.

Lamport, L. (1984). Using time instead of timeout for fault-tolerant distributed systems. ACM

Transactions on Programming Languages and Systems, 6, pp. 254–280.

Laskari, E. C., Parsopoulos, K. E., & Vrahatis, M. (2002). Particle swatn optimization for in-

teger programming. In Proceedings of the IEEE Congress on Evolutionary Computation,

Honolulu, USA, pp. 1582–1587.

Lin, J., Morse, A. S., & Anderson, B. D. O. (2003). The multi-agent rendezvous problem. In

Proceedings of the IEEE Conference on Decision and Control, Maui, USA, pp. 1508–1513.

Lin, Z., Francis, B., & Maggiore, M. (2005). Necessary and sufficient graphical conditions for

formation control of unicycles. IEEE Transactions on Automatic Control, 50, pp. 121–127.

MAGICC Lab, BYU (2010). Aviones: UAV Flight Simulator. http://aviones.

sourceforge.net/. (Accessed: 2010, May 11).

Mao, Z.-H., Feron, E., & Bilimoria, K. (2000). Stability of intersecting aircraft flows under decen-

tralized conflict avoidance rules. In AIAA Guidance, Navigation, and Control Conference,

Denver, USA. Paper No. AIAA-2000-4271.

McLain, T. W., & Beard, W. R. (2005). Coordination variables, coordination functions, and co-

operative timing missions. AIAA Journal of Guidance, Control, and Dynamics, 28, pp.

150–161.

195

McMasters, J. H., & Cummings, R. M. (2002). Airplane design - past, present, and future. Journal

of Aircraft, 39, pp. 10–17.

Merz, A. W. (1991). Maximum-miss aircraft collision avoidance. Dynamics and Control, 1, pp.

25–34.

Murphey, R. A. (1999). Target-based weapon target assignment problems. In Pardalos, P. M., &

Pitsoulis, L. S. (Eds.), Nonlinear Assignment Problems: Algorithms and Applications, pp.

39–53. Kluwer Academic Publishers, Dordrecht.

Naidu, D. S., & Calise, A. J. (2001). Singular perturbations and time scales in guidance and control

of aerospace systems. Journal of Guidance, Control, and Dynamics, 24, pp. 1057–1078.

Nemhauser, G. L., & Wolsey, L. A. (1999). Integer programming. In Nemhauser, G. L., Rinnoy

Kan, A. H. G., & Todd, M. J. (Eds.), Optimization, Vol. 1 of Handbooks in Operations

Research and Management Science. Elsevier, Amsterdam.

Nisan, N., Schapira, M., Valiant, G., & Zohar, A. (2011). Best response mechanisms. In Proceed-

ings of the 2nd Symposium on Innovations in Computer Science, Beijing, China.

Notarstefano, G., & Bullo, F. (2006). Distributed consensus on enclosing shapes and minimum

time rendezvous. In Proceedings of the IEEE Conference on Decision and Control, San

Deigo, USA, pp. 4295–4300.

Nygard, K. E., Chandler, P. R., & Pachter, M. (2001). Dynamic network flow optimization models

for air vehicle resource allocation. In Proceedings of the the American Control Conference,

Arlington, USA, pp. 1853–1858.

Obraczka, K., Viswanath, K., & Tsudik, G. (2001). Flooding for reliable multicast in multi-hop

ad hoc networks. Wireless Networks, 7, pp. 627–634.

Panyakeow, P., & Mesbahi, M. (2010). Decentralized deconfliction algorithms for unicycle UAVs.

In Proceedings of the American Control Conference, Baltimore, USA, pp. 794–799.

Papadimitriou, C. H., & Steiglitz, K. (1982). Combinatorial Optimization: Algorithms and Com-

plexity. Prentice-Hall, New Jersey.

Park, J.-W., Oh, H.-D., & Tahk, M.-J. (2008). UAV collision avoidance based on geometric

approach. In Proceedings of the SICE Annual Conference, Tokyo, Japan.

Parker, L. E., & Tang, F. (2006). Building multi-robot coalitions through automated task solution

synthesis. Proceedings of the IEEE, 94, pp. 1289–1305.

196

Parsopoulos, K. E., & Vrahatis, M. N. (2002). Recent approaches to global optimization problems

through particle swarm optimization. Natural Computing, 1, pp. 235–306.

Rahmani, A., Kosuge, K., Tsukamaki, T., & Mesbahi, M. (2008). Multiple UAV deconfliction via

navigation functions. In AIAA Guidance, Navigation, and Control Conference and Exhibit,

Honolulu, USA. Paper No. AIAA-2008-6626.

Rahwan, T., Ramchurn, S., Jennings, N., & Giovannucci, A. (2009). An anytime algorithm for

optimal coalition structure generation. Journal of Artificial Intelligence Research, 34, pp.

521–567.

Rasmussen, S., & Schumacher, C. (2007). Cooperative control of unmanned aerial vehicles. In-

ternational Journal of Robust and Nonliner Control, 18, pp. 115–117.

Reid, J. G., Furlough, T. L., & Young, J. T. (1981). Maximum acceleration design, digital optimal

guidance. In Proceedings of the AIAA Guidance and Control Conference, Albuquerque,

USA, pp. 416–422. Paper No. AIAA-1981-1782.

Ren, W., & Beard, R. W. (2008). Distributed Consensus in Multi-Vehicle Cooperative Control:

Theory and Applications. Springer-Verlag, London.

Roussos, G., Dimarogonas, D. V., & Kyriakopoulos, K. J. (2010). 3D navigation and collision

avoidance for nonholonomic aircraft-like vehicles. International Journal of Adaptive Con-

trol and Signal Processing, 24, pp. 900–920.

RTCA (2002). Minimum aviation system performance standards for automatic dependent surveil-

lance broadcast (ads-b). Tech. rep., Do-242A, RTCA.

Samek, J., Sislak, D., Volf, P., & Pechoucek, M. (2007). Multi-party collision avoidance among

unmanned aerial vehicles. In Proceedings of the International Conference on Intelligent

Agent Technology, Fremont, USA, pp. 403–406.

Sandholm, T., Larson, K., Andersson, M., Shehory, O., & Tohme, F. (1999). Coalition structure

generation with worst case guarantees. Artificial Intelligence, 111, pp. 209–238.

Scerri, P., Liao, E., Xu, Y., Lewis, M., Lai, G., & Sycara, K. (2008). Coordinating very large

groups of wide area search munitions. In Grundel, D., Murphey, R., & Pardalos, P. M.

(Eds.), Theory and Algorithms for Cooperative Systems, Vol. 4 of Series on Computers and

Operations Research, pp. 451–480. World Scientific Publishing Co. Pte. Ltd., Singapore.

Schumacher, C., & Chandler, P. (2004). UAV task assignment with timing constraints via mixed-

integer linear programming. In AIAA Unmanned Unlimited Technical Conference, Workshop

and Exhibit, Chicago, USA. Paper No. AIAA-2004-6410.

197

Shanmugavel, M., Tsourdos, A., White, B., & Zbikowski, R. (2010). Co-operative path planning

of multiple UAVs using Dubins paths with clothoid arcs. Control Engineering Practice, 18,

pp. 1084–1092.

Shanmugavel, M., Tsourdos, A., Zbikowski, R., White, B. A., Rabbath, C. A., & Lechevin, N.

(2006). A solution to simultaneous arrival of multiple uavs using pythagorean hodograph

curves. In Proceedings of the American Control Conference, Minneapolis, USA.

Shehory, O. M., & Kraus, S. (1998). Methods for task allocation via agent coalition formation.

Artificial Intelligence, 101, pp. 165–200.

Shehory, O. M., Sycara, K., & Jha, S. (1998). Multi-agent coordination through coalition for-

mation. In Singh, M. P., Rao, A., & Wooldridge, M. J. (Eds.), Intelligent Agents IV: Agent

Theories, Architectures and Languages, Vol. 1365 of Lecture Notes in Artificial Intelligence,

pp. 143–154. Springer-Verlag, Berlin.

Shi, Y., & Eberhart, R. C. (1998). A modified particle swarm optimizer. In Proceedings of the

IEEE Conference on Evolutionary Computation, Anchorage, USA, pp. 69–73.

Shiller, Z., Large, F., & Sekhavat, S. (2001). Motion planning in dynamic environments: obstacles

moving along arbitrary trajectories. In Proceedings of IEEE International Conference on

Robotics and Automation, Seoul, Korea, pp. 3716–3721.

Shim, D. H., Kim, H. J., & Sastry, S. (2003). Decentralized nonlinear model predictive control of

multiple ying robots in dynamic enviornments. In Proccedings of the IEEE Conference on

Decision and Control, Maui, USA, pp. 3621–3626.

Shima, T., Schumacher, C., & Rasmussen, S. (2009). Multiple assignments. In Shima, T., &

Rasmussen, S. (Eds.), UAV Cooperative Decision and Control: Challenges and Practical

Approaches, No. 18 in Advances in Design and Control, pp. 53–72. SIAM, Philadelphia.

Shin, H., Tsourdos, A., White, B., Shanmugavel, M., & Tahk, M. (2008). UAV conflict detection

and resolution for static and dynamic obstacles. In AIAA Guidance, Navigation, and Control

Conference and Exhibit, Honolulu, USA. Paper No. AIAA-2008-6521.

Sinha, A., & Ghose, D. (2006). Generalization of linear cyclic pursuit with application to ren-

dezvous of multiple autonomous agents. IEEE Transactions on Automatic Control, 51, pp.

1819–1824.

Sinha, A., & Ghose, D. (2007). Generalization of non linear cyclic pursuit. Automatica, 43, pp.

1954–1960.

198

Sislak, D., Rehak, M., Pechoucek, M., Pavlicek, D., & Uller, M. (2006). Negotiation-based

approach to unmanned aerial vehicles. In Proceedings of the IEEE Workshop on Distributed

Intelligent Systems: Collective Intelligence and Its Applications, Prague, Czech Republic,

pp. 279–284.

Sislak, D., Volf, P., Komenda, A., Samek, J., & Pechoucek, M. (2007). Agent-based multi-layer

collision avoidance to unmanned aerial vehicles. In Proceedings of the International Con-

ference on Integration of Knowledge Intensive Multi-Agent Systems, Waltham, USA, pp.

365–370.

Smith, A. L., & Harmon, F. G. (2010). UAS collision avoidance algorithm based on an aggregate

collision cone approach. Journal of Aerospace Engineering. DOI: 10.1061/(ASCE)AS.

1943-5525.0000081.

Stevens, B. L., & Lewis, F. L. (2003). Aircraft Control and Simulation. John Wiley and Sons,

New Jersey.

Sujit, P. B., & Ghose, D. (2009). Negotiation schemes for multi-agent cooperative search. Pro-

ceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engi-

neering, 223, pp. 791–813.

Sujit, P. B., & Ghose, D. (2010). Two agent cooperative search using game theoretical models

with endurance time constraints. Engineering Optimization, 42, pp. 617–639.

Sujit, P. B., & Ghose, D. (2011). Self assessment based decision making for multi-agent coopera-

tive search. IEEE Transactions on Automation Science and Engineering. (to appear).

Sujit, P. B., Sinha, A., & Ghose, D. (2005). Multi-UAV task allocation using team theory. In

Proceedings of the IEEE Conference on Decision and Control, and European Control Con-

ference, Seville, Spain, pp. 1497–1502.

Summers, T. H., Akella, M. R., & Mears, M. J. (2009). Coordinated standoff tracking of moving

targets: Control laws and information architectures. Journal of Guidance, Control, and

Dynamics, 32, pp. 56–69.

Suresh, M., & Ghose, D. (2010). Role of information and communication in redefining unmanned

aerial vehicle autonomous control levels. Proceedings of the Institution of Mechanical En-

gineers, Part G: Journal of Aerospace Engineering, 224, pp. 171–197.

Tang, K., & Gerla, M. (2001). MAC reliable broadcast in ad hoc networks. In Proceedings of the

IEEE Military Communications Conference, pp. 1008–1013.

199

Tiwari, A., Fung, J., Carson, J. M., Bhattacharya, R., & Murray, R. M. (2004). A framework for

lyapunov certificates for multi-vehicle rendezvous problems. In Proceedings of the Ameri-

can Control Conference, Boston, USA, pp. 5582–5587.

Tomlin, C., Pappas, G. J., & Sastry, S. (1998). Conflict resolution for air traffic management: a

case study in multi-agent hybrid systems. IEEE Transactions on Automatic Control, 43, pp.

509–521.

Tosic, P. T., & Agha, G. A. (2005). Maximal clique based distributed coalition formation for task

allocation in large-scale multi-agent systems. In Ishida, T., Gasser, L., & Nakashima, H.

(Eds.), Massively Multi-Agent Systems I, Vol. 3446 of Lecture Notes in Artificial Intelligence,

pp. 104–120. Springer-Verlag, Berlin.

Tsourdos, A., Jeyaraman, S., Shanmugavel, M., White, B. A., & Zbikowski, R. (2005). A formal

model approach for the analysis and validation of the cooperative path planning of a UAV

team. In Proceedings of the IEE Seminar on Autonomous Agents in Control, Bicester, UK,

pp. 67–73.

Tsourdos, A., White, B., & Shanmugavel, M. (2011). Cooperative Path Planning of Unmanned

Aerial Vehicles. John Wiley and Sons, West Sussex.

Tsourdos, A., Zbikowski, R., & White, B. A. (2007). Cooperative control strategies for swarm

of unmanned aerial vehicles under motion uncertainty. In Proceedings of the Institution of

Engineering and Technology Conference on Autonomous Systems, London, UK.

Vanzin, M. M., & Barber, K. S. (2006). Decentralized partner finding in multi-agent systems.

In Scerri, P., Vincent, R., & Mailler, R. (Eds.), Coordination of Large-Scale Multiagent

Systems, pp. 75–98. Springer, New York.

Vig, L., & Adams, A. J. (2005). A framework for multi-robot coalition formation. In Proceedings

of the 2nd Indian International Conference on Artificial Intelligence, Pune, India, pp. 347–

363.

Vig, L., & Adams, A. J. (2006a). Market-based multi-robot coalition formation. In Gini, M., &

Voyles, R. (Eds.), Distributed Autonomous Robotic Systems 7, pp. 227–236. Springer-Verlag,

Tokyo.

Vig, L., & Adams, A. J. (2006b). Multi-robot coalition formation. IEEE Transactions on Robotics,

22, pp. 637–649.

200

Williams, B., & Camp, T. (2002). Comparison of broadcasting techniques for mobile ad hoc

networks. In Proceedings of the 3rd ACM Interational Symposium on Mobile Ad Hoc Net-

working and Computing, Lausanne, Switzerland, pp. 194–205.

Wright, O. (1923). How we made the first flight. Aviation, 15, pp. 737–741. Reprinted from

Flying, December, 1913.

Zagi (2010) http://zagi.com/. (Accessed: 2010, May 11).

Zarchan, P. (1990). Tactical and Strategic Missile Guidance, Vol. 124 of Progress in Astronautics

and Aeronautics. AIAA, Washington, DC.

201

202

Publications from this thesis

1. Manathara, J. G., & Ghose, D. (2011). A reactive algorithm for collision avoidance among multi-

ple unmanned aerial vehicles. (manuscript under preparation).

2. Manathara, J. G., & Ghose, D. (2011). Multiple UAV collision avoidance using realistic UAV

models. Journal of Aircraft Engineering and Aerospace Technology. (under revision).

3. Manathara, J. G., & Ghose, D. (2011). Rendezvous of multiple UAVs under collision avoidance

using consensus. Journal of Aerospace Engineering. (under revision).

4. Manathara, J. G., Sujit, P. B., & Beard, R. W. (2011). Multiple UAV coalitions for a search and

prosecute mission. Journal of Intelligent and Robotic Systems, 62, pp. 125–158. DOI: 10.1007/

s10846-010-9439-2

5. George, J., & Ghose, D. (2010). Multiple UAV collision avoidance using realistic UAV models. In

Proceedings of the International Conference on Intelligent Unmanned Systems, Bali, Indonesia.

6. George, J., & Ghose, D. (2010). Multiple UAV rendezvous under collision avoidance. In Pro-

ceedings of the International Conference on Intelligent Unmanned Systems, Bali, Indonesia.

7. George, J. M., Sujit, P. B., & Sousa, J. B. (2010). Coalition formation with communication delays

and maneuvering targets. In AIAA Guidance, Navigation and Control Conference and Exhibit,

Toronto, Canada. Paper No. AIAA-2010-8422.

8. George, J. M., Sujit, P. B., Sousa, J. B., & Pereira, F. L. (2010). Coalition formation with com-

munication ranges and moving targets. In Proceedings of the American Control Conference, Bal-

timore, USA, pp. 1605–1610.

9. George, J. M., Pinto, J., Sujit, P. B., & Sousa, J. B. (2010). Multiple UAV coalition formation

strategies. In Proceedings of 9th International Conference on Autonomous Agents and Multiagent

Systems (AAMAS 2010), Toronto, Canada, pp. 1503–1504.

10. George, J., & Ghose, D. (2009). A reactive inverse PN algorithm for collision avoidance among

multiple unmanned aerial vehicles. In Proceedings of the American Control Conference, Missouri,

USA, pp. 3890–3895.

11. George, J., & Ghose, D. (2009). A 3-D reactive algorithm for collision avoidance among multiple

unmanned aerial vehicles in dense traffic situations. In Proceedings of the International Confer-

ence and Exhibition on Aerospace Engineering, Bangalore, India, pp. 1215–1222.

203

Documents

Collision avoidance and coalition formation of multiple unmanned … · 2013-02-11 · collision avoidance maneuver, in that case, is that which increases the line of sight rate between