Quantitative Assessment of Robotic Swarm Coverage

Brendon Anderson1, Eva Loeser2, Marissa Gee3, Fei Ren1, Swagata Biswas1,Olga Turanova1, Matt Haberland1∗, and Andrea L. Bertozzi1

Abstract— This paper studies a generally applicable, sensi-tive, and intuitive error metric for the assessment of roboticswarm density controller performance. Inspired by vortexblob numerical methods, it overcomes the shortcomings of acommon strategy based on discretization, and unifies othercontinuous notions of coverage. We present two benchmarksagainst which to compare the error metric value of a givenswarm configuration: non-trivial bounds on the error metric,and the probability density function of the error metric whenrobot positions are sampled at random from the target swarmdistribution. We give rigorous results that this probabilitydensity function of the error metric obeys a central limittheorem, allowing for more efficient numerical approximation.For both of these benchmarks, we present supporting theory,computation methodology, examples, and MATLAB implemen-tation code.

I. INTRODUCTION

Much of the research in swarm robotics has focused ondetermining control laws that elicit a desired group behaviorfrom a swarm [1], while less attention has been placed onmethods for quantifying and evaluating the performance ofthese controllers. Both [1] and [2] point out the lack ofdeveloped performance metrics for assessing and compar-ing swarm behavior, and [1] notes that when performancemetrics are developed, they are often too specific to the taskbeing studied to be useful in comparing performance acrosscontrollers. This paper develops an error metric that evaluatesone common desired swarm behavior: distributing the swarmaccording to a prescribed spatial density.

In many applications of swarm robotics, the swarm mustspread across a domain according to a target distribution inorder to achieve its goal. Some examples are in surveillanceand area coverage [3]–[6], achieving a heterogeneous targetdistribution [7]–[11], and aggregation and pattern formation[12]–[15]. Despite the importance of assessing performance,some studies such as [10] and [15] rely only on qualitativemethods such as visual comparison. Others present perfor-mance metrics that are too specific to be used outside of thespecific application, such as measuring cluster size in [12],distance to a pre-computed target location in [6], and areacoverage by tracking the path of each agent in [3]. In [14] anL2 norm of the difference between the target and achieved

*Corresponding author: haberland@ucla.edu1UCLA, Dept. of Mathematics, Los Angeles, CA 900952Brown University, Mathematics Department, Providence, RI 029123Harvey Mudd College, Dept. of Mathematics, Claremont, CA 91711The authors gratefully acknowledge the support of NSF grant CMMI-

1435709, NSF grant DMS-1502253, the Dept. of Mathematics at UCLA,and the Dept. of Mathematics at Harvey Mudd College. The authorsalso thank Hao Li (UCLA) for his insightful contributions regarding theconnection between this error metric and kernel density estimation.

swarm densities is considered, but the notion of achievedswarm density is particular to the controllers under study.

We develop and analyze an error metric that quantifies howwell a swarm achieves a prescribed spacial distribution. Ourmethod is is independent of the controller used to generatethe swarm distribute, and thus has the potential to be usedin a diverse range of robotics applications. In [16] and [17],error metrics similar to the one presented here are used,but their properties are not discussed in sufficient detail forthem to be widely adopted. In particular, although the errormetric that we study always takes values somewhere between0 and 2, these values are, in general, not realistic for anarbitrary desired distribution and a fixed number of robots.How then, in general, is one to judge whether the value ofthe error metric, and thus the robot distribution, achieved bya given swarm control law “good” or not? We address thisby studying two benchmarks,

1) the global extrema of the error metric, and2) the probability density function (PDF) of the error

metric when robot positions are sampled from thetarget distribution,

which were first proposed in [16]. Using tools from nonlinearprogramming for (1) and rigorous probability results for (2),we put each of these benchmarks on a firm foundation.In addition, we provide MATLAB code for performing allcalculations at https://git.io/v5ytf. Thus, by usingthe methods developed here, one can assess the performanceof a given controller by comparing the error metric valueof robot configurations produced by the controller againstbenchmarks (1) and (2).

Our paper is organized as follows. Our main definition, itsbasic properties, and a comparison to common discretizationmethods in Section II. Then, Section III and Section IV aredevoted to studying (1) and (2), respectively. We suggestfuture work and conclude in Section V.

II. QUANTIFYING COVERAGE

One difficulty in quantifying swarm coverage is that thetarget density within the domain is often prescribed as acontinuous (or piecewise continuous) function, yet the swarmis actually composed of a finite number of robots at discretelocations. Although a common approach for comparing theactual positions of robots to the target density function beginsby discretizing the domain (e.g. [8], [9]), this is not theroute we take here. (We compare the two methods directlyin Subsection II-E.) The method we present and analyze isinspired by vortex blob numerical methods for the Eulerequation and the aggregation equation (see [18] and the

references therein.) In addition, a similar strategy was usedin [16] and [17] to measure the effectiveness of a certainrobotic control law. Our definition of error metric below isthe time-independent version of that in [16].

A. Definition

We are given a bounded region1 Ω ∈ Rd, a desired robotdistribution ρ : Ω → (0,∞) satisfying

∫Ωρ(x) dx = 1, and

N robot positions x1, ..., xN ∈ Ω. To compare the discretepoints x1, ...xN to the function ρ, we place a “blob” ofsome shape and size at each point xi. The shape and sizeparameters have two physical interpretations as:• the area in which an individual robot performs its task,

or• inherent uncertainty in the robot’s position.

Either of these interpretations (or a combination of thetwo) can be invoked to make a meaningful choice of theseparameters.

The blob or robot blob function can be any function K(z) :Rd → R that is non-negative on Ω and satisfies

∫Rd K = 1.

It is also natural to require that K is radially symmetricand decreasing along radial directions. We need one moreparameter, a positive number δ, that controls how far theeffective area (or inherent positional uncertainty) of the robotextends. We then define Kδ as,

Kδ(z) =1

δdK(zδ

). (II.1)

We point out that we still have∫Rd K

δ(z) dz = 1. One choiceof Kδ would be a scaled indicator function, for instance, afunction of constant value within a disc of radius δ and 0elsewhere. This is an appropriate choice when a robot isconsidered to either perform its task at a point or not, andwhere there is no notion of the degree of its effectiveness.For the remainder of this paper, however, we usually take Kto be the Gaussian

G(z) =1

2πexp

(−|z|

2

2

),

which is useful when the robot is most effective at its tasklocally and to a lesser degree some distance away. To definethe swarm blob function ρδN (x), we place a blob Gδ at eachrobot position xi, sum over i and renormalize, yielding,

ρδN (z) =

∑Ni=1G

δ(z − xi)∑Ni=1

∫ΩGδ(z − xi) dz

. (II.2)

This swarm blob function gives a continuous representationof how the discrete robots are distributed. Note that eachintegral in the denominator of (II.2) approaches 1 if δ issmall or all robots are far from the boundary, so that wehave,

ρδN (z) ≈ 1

N

N∑i=1

Gδ(z − xi). (II.3)

1We present our definitions for any number of dimensions d ≥ 1 todemonstrate their generality. However, in the latter sections of the paper, werestrict ourselves to d = 2, a common setting in ground-based applications.

We now define our notion of error, which we refer to asthe error metric and denote by eδN :

eδN =

∫Ω

∣∣ρδN (z)− ρ(z)∣∣ dz. (II.4)

We sometimes write this as eδN (x1, ..., xN ) to emphasize thedependence on the robot positions.

B. Remarks and Basic Properties

Our error is defined as the L1 norm between the swarmblob function and the desired robot distribution ρ. One coulduse another Lp norm; however, p = 1 is a standard choice inapplications that involve particle transportation and coveragesuch as [17]. Moreover, the L1 norm has a key property: forany two integrable functions f and g,∫

Ω

|f − g| dx = 2 supB⊂Ω

∣∣∣∣∫B

f −∫B

g

∣∣∣∣ .The other Lp norms do not enjoy this property [19, Chapter1]. Consequently, by measuring L1 norm on Ω, we are alsobounding the error we make on any particular subset, and,moreover, knowing the error on “many” subsets gives an esti-mate of the total error. This means that by using the L1 normwe capture the idea that discretizing the domain providesa measure of error, but avoid the pitfalls of discretizationmethods described in Subsection II-E.

Studies in optimal control of swarms often use the L2

norm due to the favorable inner product structure [17]. Wepoint out that the L1 norm is bounded from above by the L2

norm due to the Cauchy-Schwarz inequality and the fact thatΩ is a bounded region. Thus, if an optimal control strategycontrols the L2 norm, then it will also control the error metricwe present here.

Last, we note:

Proposition II.1. For any Ω, ρ, δ, N , and (x1, ..., xN ),

0 ≤ eδN ≤ 2.

Proof. This follows directly from the basic property∫|f | −

∫|g| ≤

∫|f − g| ≤

∫|f |+

∫|g|

and our normalization.

The theoretical minimum of eδN can only be approachedfor a general target distribution when δ is small and N islarge, or in the trivial special case when the target distributionis exactly the sum of N Gaussians of the given δ, motivatingthe need to develop benchmarks (1) and (2).

C. Variants of the Error Metric

The notion of error defined by (II.4) is suitable for tasksthat require good instantaneous coverage. For tasks thatinvolve tracking average coverage over some period of time(and in which the robot positions are functions of time t),an alternative “cumulative” version of the error metric is∫

Ω

∣∣∣∣∣∣ 1

M

M∑j=1

ρδN (z, tj)− ρΩ(z)

∣∣∣∣∣∣ dz (II.5)

for time points j = 1, . . . ,M . This is a practical, discrete-time version of the metric used in [17], which uses atime integral rather than a sum, as in practice, positionmeasurements can only be made at discrete times. Whilethis cumulative error metric is, in general, distinct from theinstantaneous version of (II.4), note that the extrema and PDFof this cumulative version can be calculated as the extremaand PDF of the instantaneous error metric with MN robots.Therefore, in subsequent sections we restrict our attentionto the extrema and PDF of the instantaneous formulationwithout loss of generality.

In addition, [17] considers a one-sided notion of error, inwhich a scarcity of robots is penalized but an excess is not,that is

eδN =

∫Ω−

∣∣ρδN (z)− ρ(z)∣∣ dz,

where Ω− := z|ρδN (z) ≤ ρ(z). Remarkably, e and eδN arerelated by:

Proposition II.2. eδN = 2eδN .

Proof. Let Ω+ = Ω \ Ω−. Since Ω = Ω− ∪ Ω+, we have,∫Ω−

ρδN +

∫Ω+

ρδN =

∫Ω

ρδN = 1 =

∫Ω

ρ =

∫Ω−

ρ+

∫Ω+

ρ.

Rearranging and taking absolute values we find∫Ω−

∣∣ρδN − ρ∣∣ =

∫Ω+

∣∣ρδN − ρ∣∣ ,as each integrand is of the same sign everywhere within thelimits of integration. We notice that the left-hand side andtherefore the right-hand side of the previous line equal eδN .On the other hand, their sum equals eδN . Thus our claimholds.

The definition of eδN is particularly useful in conjunctionwith the choice of Kδ as a scaled indicator function, as eδNbecomes a direct measure of the deficiency in coverage of arobotic swarm. For instance, given a swarm of surveillancerobots, each with observational radius δ, eδN is the percentageof the domain not observed by the swarm.2 Proposition II.2implies that 1

2eδN also enjoys this interpretation.

D. Calculating eδNIn practice, the integral in (II.4) can rarely be carried

out analytically, primarily because the integral needs to beseparated into regions for which the quantity ρδN (z)−ρ(z) ispositive and regions for which it is negative, the boundariesbetween which are usually difficult to express in closedform. We find that a simple generalization of the familiarrectangle rule converges linearly in dimensions d ≤ 3 andexpect Monte Carlo and Quasi-Monte Carlo methods toproduce a reasonable estimate in higher dimensions3. Moreadvanced quadrature rules can be used in low dimensions, but

2The notion of ‘coverage’ in [3] might be interpreted as e with δ as thewidth of the robot. There, only the time to complete coverage (e = 0) wasconsidered.

3We look forward to a physical swarm of robots being deployed - andthese results employed - in four dimensions!

may suffer in accuracy due to nonsmoothness in the targetdistribution and/or stemming from the absolute value takenwithin the integral.

E. The Pitfalls of Discretization

We conclude this section by analyzing a measure of errorthat involves discretizing the domain. In particular, we showin Propositions II.3 and II.4 that the values produced by thismethod are strongly dependent on a choice of discretizationthat is not readily made based on physical arguments. Inparticular, this error approaches its theoretical minimumwhen the discretization is too coarse and its theoreticalmaximum when the discretization is too fine (regardless ofrobot positions).

Discretizing the domain means dividing Ω into M disjointregions Ωi ⊂ Ω such that

⋃Mi=1 Ωi = Ω. Within each region,

the desired proportion of robots is the integral of the targetdensity function within the region

∫Ωiρ(x)dx. Using Ni to

denote the observed number of robots in Ωi, we can definean error metric as

µ =

M∑i=1

∣∣∣∣∫Ωi

ρ(x) dx− NiN

∣∣∣∣ . (II.6)

It is easy to check that 0 ≤ µ ≤ 2 always holds. Oneadvantage of this approach is that µ is very easy to compute,but there are two major drawbacks.

1) Choice of Domain Discretization: The choice fordomain discretization is not unique, and this choice candramatically affect the value of µ, as demonstrated by thefollowing two propositions.

Proposition II.3. If M = 1 then µ = 0.

Proof. When M = 1, (II.6) becomes,

µ = |∫

Ω

ρ(x) dx− 1| = 0.

The situation of perfectly coarse discretization is in com-plete contrast.

Proposition II.4. Suppose the robot positions are distinct4

and the regions Ωi are sufficiently small such that, for eachi, Ωi contains at most one robot and

∫Ωiρ ≤ 1/N holds.

Then µ→ 2 as |Ωi| → 0.

Proof. Let us relabel the Ωi so that for i = 1, ...,M − Nthere is no robot in Ωi, and thus each of the Ωi for i =M −N + 1, ...,M contains exactly one robot. In this case,the expression for error µ becomes,

µ =

M−N∑i=1

∫Ωi

ρ+

M∑i=M−N+1

∣∣∣∣∫Ωi

ρ− 1

N

∣∣∣∣ . (II.7)

4This is reasonable in practice as two physical robots cannot occupy thesame point in space. In addition, the proof can be modified to produce thesame result even if the robot positions coincide.

Since∫

Ωiρ ≤ 1/N holds, then with the identity

M−N∑i=1

∫Ωi

ρ = 1−M∑

i=M−N+1

∫Ωi

ρ,

we can rewrite (II.7) as,

µ = 1−M∑

i=M−N+1

∫Ωi

ρ+

M∑i=M−N+1

(1

N−∫

Ωi

ρ

)

= 2− 2

M∑i=M−N+1

∫Ωi

ρ.

Thus µ→ 2 as M →∞ and |Ωi| → 0.

Note that the shape of each region is also a choice thatwill affect the calculated value. While our approach alsorequires the choice of some size and shape (namely, δ andK), these parameters have much more immediate physicalinterpretation, making appropriate choices easier to make.

2) Error Metric Discretization and Desensitization: Per-haps more importantly, by discretizing the domain, we alsodiscretize the range of values that the the error metric canassume. While this may not be inherently problematic, wehave simultaneously desensitized the error metric to changesin robot distribution within each region. That is, as long asthe number of robots Ni within each region Ωi does notchange, the distribution of robots within any and all Ωi maybe changed arbitrarily without affecting the value of µ. Onthe other hand, the error metric eδN is continuously sensitiveto differences in distribution.

III. ERROR METRIC EXTREMA VIA NONLINEARPROGRAMMING

As noted above, eδN is a function of the robot positions(x1, ..., xN ). We are interested in finding or approximatingits extrema. However, it is a highly nonlinear function, andtrying to find its extrema analytically has been intractable.Thus, we approach this problem by using a standard nonlin-ear programming solver, MATLAB’s fmincon.

A limitation of all general nonlinear programming algo-rithms is that successful termination produces only a localminimum, which is not guaranteed to be the global minimum.There is no obvious re-formulation of this problem for whicha global solution is guaranteed, so the best we can do is to usea local minimum produced by nonlinear programming as anupper bound for the minimum of the error metric. Heuristics,such as multi-start (running the optimization many timesfrom several initial guesses and taking the minimum of thelocal minima) can be used to make this bound tighter. Thisbound serves as a benchmark against which we can comparean achieved value of the error metric. This is reasonable, asif a configuration of robots with a lower value of the errormetric exists but eludes numerical optimization, it is probablynot a fair standard against which to compare the performanceof a general controller.

To bound the global minimum, we first define the non-linear optimization problem of interest. We focus on Ω ⊂

R2. Let x = (x1, ..., xN ) represent a vector of N robotcoordinates. The optimization problem is,

minimize eδN (x1, ..., xN ), (III.1)subject to xi ∈ Ω for i ∈ 1, 2, . . . , N.

Note that the same problem structure can be used to find themaximum of the error metric by minimizing the negative ofeδN .

A. Example

Using the “ring distribution” from [16]5 with a swarm ofN = 200 robots with δ = 2in (the physical radius of therobots in that study), we computed 50 local minima of theerror metric starting with random initial guesses. We tookthe lowest of these local minima as an upper bound on theglobal minimum of the error metric. These results are shownin Figure 1; equivalent results for the maximum are in Figure2. Note that the minimum of the error metric eδN = 0.28205was significantly higher than zero for this finite number ofrobots of nonzero radius.

The performance of a robot distribution controller can bequantitatively assessed by calculating the error value eobservedof a robot configuration it produces, and comparing the valueagainst the extrema bounds e− and e+. Consider the quantity

eobserved − e−

e+ − e−: (III.2)

if the highest observed value of this ratio is 10%, theperformance of the controller is quite close to the bestpossible, whereas if this ratio is typically 90% or higher,the performance of the controller is rather poor.

Fig. 1. Swarm blob function ρδ=2inN=200 corresponding with the robot

distribution that yields a minimum value of the error metric for the ringdistribution, 0.28205.

5The ring distribution ρring is defined on the Cartesian plane withcoordinates z = (z1,z2) as follows. Let inner radius r1 = 11.4in,outer radius r2 = 20.6in, width w = 48in, height h = 70in, andρ0 = 2.79 × 10−5. Let domain Ω = z : z1 ∈ [0, w], z2 ∈ [0, h]and region Γ = z : r21 < (z1 − w

2)2 + (z2 − h

2)2 < r22. Then

ρring(z) = 36ρ0 if z ∈ Ω ∩ Γ, ρ0 if z ∈ Ω \ Γ.

Fig. 2. Swarm blob function ρδ=2inN=200 corresponding with the robot

distribution that yields a maximum value of the error metric for the ringdistribution, 1.9867. This occurs when all robots coincide outside the ring.

B. Error Metric for Optimal Swarm Design

So far we have taken N and δ to be fixed; we haveassumed that the robotic agents and size of the swarm havealready been chosen. We briefly consider the use of the errormetric as an objective function for the design of a swarm.Simply adding δ > 0 as a decision variable to (III.1) andsolving at several fixed values of N provides insight intohow many robots of what effective working radius are neededto achieve a given level of coverage for a particular targetdistribution. Visualizations of such calculations are providedin Figure 3 and the supplemental video.

Note that ‘breakthroughs’, or relatively rapid decreases inthe error metric, can occur once a critical number of robotsare available; these correspond with a conceptual changein the distribution of robots. For example, at N = 22 therobots are arranged in a single ring; beginning with N = 25we see the robots begin to be arranged in two separateconcentric rings of different radii and the error metric beginsto drop sharply. On a related note, there are also ‘lulls‘ inwhich increasing the number of robots has little effect on theminimum value of the error metric , such as between N = 44and N = 79. Studies like these can help a swarm designerdetermine the best number of robots N and effective radiusof each δ to achieve the required coverage.

IV. ERROR METRIC PROBABILITY DENSITY FUNCTION

In the previous section we have described how to findbounds on the minimum and maximum values for error. Aquestion that remains is, how “easy” or “difficult” is it toachieve such values? Answering this question is importantin order to use the error metric to assess the effectiveness ofan underlying control law. Indeed, a given control law maytend to produce robot positions with error well above theminimum, and it is necessary to assess these values as well.

According to the setup of our problem, the goal of anysuch control law is to get the robots to achieve the desireddistribution ρ. Thus, whatever the particular control law is,

it is natural to compare its outcome to simply picking therobot positions at random from the target distribution ρ.In this section we consider the robot positions as beingsampled directly from the desired distribution and study thestatistical properties of the error metric in this situation,both analytically and numerically. In Subsection IV-A wepresent rigorous results that show that the error metric hasan approximately normal distribution in this case. As acorollary we obtain that the limit of this error is zero asN approaches infinity and δ approaches 0. Subsections IV-Band IV-C include a numerical demonstration of these results.In addition, in IV-C, we suggest how the PDF may serve asa benchmark with which to evaluate the performance of aswarm distribution controller.

The theoretical results presented in the next subsectionnot only support our numerical findings, they also allow forfaster computation. Indeed, if one did not already know thatthe error when robots are sampled randomly from ρ hasa normal distribution for large N , tremendous computationmay be needed to get an accurate estimate of the cumulativedistribution function (CDF) and probability density function(PDF). On the other hand, since the results we present provethat the error metric has a normal distribution for large N ,only a sample mean and variance need to be calculated inorder to approximate the entire CDF and PDF.

We take the robots’ positions X1, . . . , XN , to be inde-pendent, identically distributed bivariate random vectors inΩ ⊂ R2 with probability density function ρ. We place a blobof shape K at each of the Xi (previously we took K to bethe Gaussian G), so that the swarm blob function is,

ρδN (z) =1

Nδ2

N∑i=1

Kδ (z −Xi) , (IV.1)

where Kδ is defined by (II.1). We point out that the right-hand side of (IV.1) is exactly that of (II.3) upon taking K tobe the Gaussian G and the robot locations xi to be the ran-domly selected Xi. The error eδN is now a random variable,the value of which depends on the particular realization ofthe robot positions X1, . . . , XN .

A. Theoretical Central Limit Theorem

The expression (IV.1) is the so-called kernel density esti-mator of ρ. This arises in statistics, where ρ is thought ofas unknown, and ρδN is considered as an approximation toρ. It turns out that, under appropriate hypotheses, the L1

error between ρ and ρδN has a normal distribution with meanand variance that approach zero as N approaches infinity. Inother words, a central limit theorem holds for the error. Forsuch a result to hold, δ and N have to be compatible. Thus,for the remainder of this subsection δ will depend on N , andwe display this as δ(N). We have,

Theorem IV.1. Suppose ρ is continuously twice differen-tiable, K is zero outside of some bounded region and radiallysymmetric. Then, for δ(N) satisfying

δ(N) = O(N−1/6) and limN→∞

δ(N)N1/4 =∞, (IV.2)

Fig. 3. Swarm blob functions ρδN corresponding with the robot distributions and values of δ that yield the minimum value of the error metric forthe ring distribution target. Inset graph shows the relationship between N and the minimum value of the error metric observed from repeated numericaloptimization. Due to long execution time of optimization at N = 256, fewer local minima were calculated; this may explain the rise in the minimal errormetric value.

we have

eδ(N)N ≈ N

(e(N)

N1/2,σ(N)2δ(N)2

N

),

where σ2(N) and e(N) are deterministic quantities that arebounded uniformly in N .6

Proof. This follows from Horvath [20, Theorem, page 1935].For the convenience of the reader, we record that the quan-tities N , δ, ρ, ρδN , eδN that we use here correspond to n, h,

6Here N (µ, σ2) denotes the normal random variable of mean µ andvariance σ2, and we use the notation ≈ to mean that the difference of thequantity on the left-hand side and on the right-hand side converges to zeroin the sense of distributions.

f , fn, In in [20]. We do not present the exact expressionsfor σ(N) and e(N); they are written in [20, page 1934].The uniform boundedness of σ is exactly line (1.2) of [20];the boundedness for e(N) is not written explicitly in [20]so we briefly explain how to derive it. In the expression fore(N) in [20], mN is the only term that depends on N . Astandard argument that uses the Taylor expansion of ρ andthe symmetry of the kernel K (see, for example, Section2.4 of the lecture notes [21]) yields that mN is uniformlybounded in N .

From this it is easy to deduce:

Corollary IV.2. Under the hypotheses of Theorem IV.1, the

error eδ(N)N converges in distribution to zero.

Remark IV.3. There are a few ways in which practicalsituations may not align perfectly with the assumptions of[20]. However, we posit that in all of these cases, thedifference between these situations and that studied in [20]is numerically insignificant. We now briefly summarize thesethree discrepancies and indicate how to resolve them.

First, we defined our density ρδN by (II.2), but in thissection we use a version with denominator N . However, asexplained above, the two expressions approach each otherfor small δ, and this is the situation we are interested inhere. Second, a ρ that is piecewise continuous like the ringdistribution is not twice differentiable. We point out that anarbitrary density ρ may be approximated to arbitrary preci-sion by a smoothed out version, for example by convolutionwith a mollifier (a standard reference is Brezis [22, Section4.4]). Third, in our computations we use the kernel G,which is not compactly supported, for the sake of simplicity.Similarly, this kernel can be approximated, with arbitraryaccuracy, by a compactly supported version. Making thesechanges to the kernel or target density would not affect theconclusions of numerical results.

B. Numerical Approximation of the Error Metric PDF

In this subsection we describe how to numerically find theCDF and PDF of eδN when the robot positions are randomlysampled from ρ. We first establish:

Proposition IV.4. Let FeδN denote the CDF of eδN when therobot positions are randomly sampled from ρ. Then,

FeδN (z) =

∫ΩN

1x|eδN (x)≤z

N∏i=1

ρ(xi)dx. (IV.3)

Proof. We recall a basic probability fact. Let Y be a randomvector with values in A ⊂ RD with probability densityfunction f , and let g be a real-valued function on Rd. TheCDF for g(Y ), denoted Fg(Y ), is given by,

Fg(Y )(z) = P(g(Y ) ≤ z) =

∫A

1y|g(y)≤zf(y)dy, (IV.4)

where 1 denotes the indicator function.In our situation, we take the random vector Y to be X :=

(X1, ..., XN ). Since X takes values in ΩN := Ω × ... × Ω,we take A to be ΩN (we point out that here D = 2N ). Sinceeach Xi has density ρ, the density of X is the function ρ,defined by,

ρ(x1, ..., xn) =

N∏i=1

ρ(xi).

Thus, taking f and g in (IV.4) to be ρ and eδN , respectively,yields (IV.3).

Notice that since each of the xi is itself a 2-dimensionalvector (the Xi are random points in the plane and we areusing the notation x = (x1, ..., xN )), the integral definingthe cumulative distribution function of the error metric isof dimension 2N . Finding analytical representations for the

CDF is combinatorially complex and quickly becomes in-feasible for large swarms. Therefore, we approximate (IV.3)using Monte Carlo integration, which is well-suited for high-dimensional integration [23]7, fit a Gauss Error Function tothe data, then differentiate to obtain the PDF. We remark thatwe have used the notation of an indicator function above inorder to express the quantity of interest in a way that is easilyapproximated with Monte Carlo integration.8

C. Example

The CDF for the ring distribution with 200 robots usingM = 1000 Monte Carlo evaluation points is shown by asolid gray line in Figure 4. The numerical approximationappears to closely match a Gauss Error Function (erf(·), theintegral of a Gaussian G) as theory predicts. Therefore ananalytical erf(·) curve, represented by the dashed line, is fit tothe data using MATLAB’s least squares curve fitting routinelsqcurvefit. To obtain the probability density functionof the error metric, the analytical curve fit for the CDF isdifferentiated, and the result is also shown in Figure 4.

If a significant fraction of the configurations producedby a given controller yield an error metric value that lieswithin the middle quartiles of this error metric PDF, thenthe performance appears consistent with that expected of astochastic swarm controller such as that of [16]. We suggestthat the error metric values produced by a deterministiccontroller should lie along the lower tail of the error metricPDF to be considered “good”.

Fig. 4. The CDF of the error metric when robot positions are sampledfrom ρ is approximated by Monte Carlo integration, an erf curve fit matchesclosely, and the PDF is taken as the derivative of the fitted erf .

V. FUTURE WORK AND CONCLUSION

While the concepts presented herein are expected to besufficient for the comparison and evaluation of swarm distri-bution controllers, the computational techniques are certainlyopen to analysis and improvement. For instance, is therea simpler, deterministic method of approximating the error

7Quasi-Monte Carlo techniques, which use a low-discrepancy sequencerather than truly random evaluation points, promise somewhat faster con-vergence but require considerably greater effort to implement. The difficultyis in generating a low-discrepancy sequence from the desired distribution,which is possible using the Hlawka-Muck method, but computationallyexpensive [24].

8Optionally, one could use random sampling to estimate only the meanand standard deviation of the Gaussian PDF, but we chose to estimate theentire CDF to show that it matches the predictions of theory.

metric PDF? Is there a more appropriate formulation fordetermining the extrema of the error metric for a given situa-tion, one that is guaranteed to produce a global optimum? Ifnot, which nonlinear programming algorithm is most suitablefor solving (III.1)? In practice, what method of quadratureconverges most efficiently to approximate the error metric?As the size of practical robot swarms will likely grow fasterthan processor speeds will increase, improved computationaltechniques will be needed to keep benchmark computationspractical.

Also, a very important question about the nature of theerror metric remains. The blob shape Kδ has an intuitivephysical interpretation, and so a reasonable choice is typ-ically easy to make. The value of the error metric for aparticular situation is certainly affected by the choice ofblob shape K and radius δ, but so are the values of theproposed benchmarks: the extrema and the error PDF. Arequalitative conclusions made by comparing the performanceof the controller to these benchmarks likely to be affectedby the choice of blob?

Open questions notwithstanding, the error metric presentedherein is sufficiently general to be used in quantifying thethe performance of one of the most fundamental tasks ofa robotic swarm controller: achieving a prescribed densitydistribution. The error metric is sensitive enough to comparethe effectiveness of given control laws for achieving a giventarget distribution. The error metric parameters, blob shapeand radius, have intuitive physical interpretations so thatthey can be chosen appropriately. Should a designer wishto interpret the performance of a given controller withoutcomparing against results of another controller, we providetwo benchmarks that can be applied to any situation: extremaof the error metric, and the probability density function ofthe error metric when swarm configurations are sampledfrom the target distribution. Using the provided code, thesemethods can easily be used to quantitatively assess theperformance of new swarm controllers and thereby improvethe effectiveness of practical robot swarms.

REFERENCES

[1] M. Brambilla, E. Ferrante, M. Birattari, and M. Dorigo, “Swarmrobotics: a review from the swarm engineering perspective,” SwarmIntelligence, vol. 7, no. 1, pp. 1–41, 2013.

[2] Y. U. Cao, A. S. Fukunaga, and A. Kahng, “Cooperative mobilerobotics: Antecedents and directions,” Autonomous robots, vol. 4,no. 1, pp. 7–27, 1997.

[3] D. J. Bruemmer, D. D. Dudenhoeffer, M. D. McKay, and M. O.Anderson, “A robotic swarm for spill finding and perimeter formation,”Idaho National Engineering and Environmental Lab, Idaho Falls, Tech.Rep., 2002.

[4] H. Hamann and H. Worn, “An analytical and spatial model of foragingin a swarm of robots,” in International Workshop on Swarm Robotics.Springer, 2006, pp. 43–55.

[5] A. Howard, M. J. Mataric, and G. S. Sukhatme, “Mobile sensornetwork deployment using potential fields: A distributed, scalable so-lution to the area coverage problem,” Distributed autonomous roboticsystems, vol. 5, pp. 299–308, 2002.

[6] M. Schwager, J. McLurkin, and D. Rus, “Distributed coverage controlwith sensory feedback for networked robots.” in robotics: science andsystems, 2006.

[7] K. Elamvazhuthi, C. Adams, and S. Berman, “Coverage and fieldestimation on bounded domains by diffusive swarms,” in Decisionand Control (CDC), 2016 IEEE 55th Conference on. IEEE, 2016,pp. 2867–2874.

[8] S. Berman, V. Kumar, and R. Nagpal, “Design of control policiesfor spatially inhomogeneous robot swarms with application to com-mercial pollination,” in Robotics and Automation (ICRA), 2011 IEEEInternational Conference on. IEEE, 2011, pp. 378–385.

[9] N. Demir, U. Eren, and B. Acıkmese, “Decentralized probabilisticdensity control of autonomous swarms with safety constraints,” Au-tonomous Robots, vol. 39, no. 4, pp. 537–554, 2015.

[10] W.-M. Shen, P. Will, A. Galstyan, and C.-M. Chuong, “Hormone-inspired self-organization and distributed control of robotic swarms,”Autonomous Robots, vol. 17, no. 1, pp. 93–105, 2004.

[11] K. Elamvazhuthi and S. Berman, “Optimal control of stochasticcoverage strategies for robotic swarms,” in Robotics and Automation(ICRA), 2015 IEEE International Conference on. IEEE, 2015, pp.1822–1829.

[12] O. Soysal and E. Sahin, “A macroscopic model for self-organizedaggregation in swarm robotic systems,” in International Workshop onSwarm Robotics. Springer, 2006, pp. 27–42.

[13] W. M. Spears, D. F. Spears, J. C. Hamann, and R. Heil, “Distributed,physics-based control of swarms of vehicles,” Autonomous Robots,vol. 17, no. 2, pp. 137–162, 2004.

[14] J. H. Reif and H. Wang, “Social potential fields: A distributedbehavioral control for autonomous robots,” Robotics and AutonomousSystems, vol. 27, no. 3, pp. 171–194, 1999.

[15] K. Sugihara and I. Suzuki, “Distributed algorithms for formationof geometric patterns with many mobile robots,” Journal of FieldRobotics, vol. 13, no. 3, pp. 127–139, 1996.

[16] H. Li, C. Feng, H. Ehrhard, Y. Shen, B. Cobos, F. Zhang, K. Elam-vazhuthi, S. Berman, M. Haberland, and A. L. Bertozzi, “Decentral-ized stochastic control of robotic swarm density: Theory, simulation,and experiment,” in Intelligent Robots and Systems (IROS), 2017IEEE/RSJ International Conference on. IEEE, 2017.

[17] F. Zhang, A. L. Bertozzi, K. Elamvazhuthi, and S. Berman, “Perfor-mance bounds on spatial coverage tasks by stochastic robotic swarms,”IEEE Transactions on Automatic Control, 2017.

[18] K. Craig and A. Bertozzi, “A blob method for the aggregationequation,” Mathematics of computation, vol. 85, no. 300, pp. 1681–1717, 2016.

[19] L. Devroye and L. Gyorfi, Nonparametric density estimation: the L1

view. Wiley, 1985.[20] L. Horvath, “On Lp-norms of multivariate density estimators,” The

Annals of Statistics, pp. 1933–1949, 1991.[21] B. E. Hansen, “Lecture notes on nonparametrics,” Lecture notes, 2009.[22] H. Brezis, Functional analysis, Sobolev spaces and partial differential

equations. Springer Science & Business Media, 2010.[23] I. Sloan, “Integration and approximation in high dimensions–a tuto-

rial,” Uncertainty Quantification, Edinburgh, 2010.[24] J. Hartinger and R. Kainhofer, “Non-uniform low-discrepancy se-

quence generation and integration of singular integrands,” in MonteCarlo and Quasi-Monte Carlo Methods 2004. Springer, 2006, pp.163–179.