Using abstractions to solve opportunistic crime securitygames at scale

Chao Zhang, Victor Bucarey∗, Ayan Mukhopadhyay†, Arunesh Sinha, Yundi Qian,Yevgeniy Vorobeychik†, Milind Tambe

University of Southern California, Los Angeles, CA, 90089, USA∗Universidad de Chile,Santiago, Region Metropolitana, Chile

†Vanderbilt University, Nashville, TN 37235, USA{zhan661, aruneshs, yundi.qian, tambe}@usc.edu, ∗vbucarey@ing.uchile.cl,

†{ayanmukg, eug.vorobey}@gmail.com

ABSTRACTIn this paper, we aim to deter urban crime by recommending opti-mal police patrol strategies against opportunistic criminals in largescale urban problems. While previous work has tried to learn crim-inals’ behavior from real world data and generate patrol strategiesagainst opportunistic crimes, it cannot scale up to large-scale urbanproblems. Our first contribution is a game abstraction frameworkthat can handle opportunistic crimes in large-scale urban areas. Inthis game abstraction framework, we model the interaction betweenofficers and opportunistic criminals as a game with discrete targets.By merging similar targets, we obtain an abstract game with fewertotal targets. We use real world data to learn and plan against op-portunistic criminals in this abstract game, and then propagate theresults of this abstract game back to the original game. Our sec-ond contribution is the layer-generating algorithm used to mergetargets as described in the framework above. This algorithm ap-plies a mixed integer linear program (MILP) to merge similar andgeographically neighboring targets in the large scale problem. Asour third contribution, we propose a planning algorithm that rec-ommends a mixed strategy against opportunistic criminals. Finally,our fourth contribution is a heuristic propagation model to handlethe problem of limited data we occasionally encounter in large-scale problems. As part of our collaboration with local police de-partments, we apply our model in two large scale urban problems:a university campus and a city. Our approach provides high predic-tion accuracy in the real datasets; furthermore, we project signifi-cant crime rate reduction using our planning strategy compared tocurrent police strategy.

Categories and Subject DescriptorsI.2.11 [Artificial Intelligence]: Distributed Artificial Intelligence

General TermsSecurity, Human Factors

KeywordsSecurity Games; Abstraction; Machine Learning

Appears in: Proceedings of the 15th International Conference on Au-tonomous Agents and Multiagent Systems (AAMAS 2016),J. Thangarajah, K. Tuyls, C. Jonker, S. Marsella (eds.),May 9–13, 2016, Singapore.Copyright c© 2016, International Foundation for Autonomous Agents andMultiagent Systems (www.ifaamas.org). All rights reserved.

1. INTRODUCTIONManaging urban crime has always posed a significant challenge

for modern society. Distinct from elaborately planned terroristsattacks, urban crimes are usually committed by opportunistic crim-inals who are less careful in planning the attack and more flexiblein executing such plans [25]. Almost universally, preventive policepatrolling is used with the goal of deterring these crimes. At thesame time, opportunistic criminals observe the police deploymentand react opportunistically . Therefore, it is very important to de-ploy the police resources strategically against informed criminals.

Previous work has tackled the problem of allocating police re-sources against opportunistic criminals. There are two approachesto recommend patrol strategies. The first approach is securitygames, such as Stackelberg Security Games [26] and OpportunisticSecurity Games [27], where the interaction between police officersand opportunistic criminals is modeled as a leader-follower game.Security games contain various extensions to handle different realworld scenarios, but the models of adversary behavior are based onexpert hypotheses, and lack detail as they are not learned from real-world data for defender’s strategy and adversary’s reaction. Thesecond approach uses larger amounts of data, such as the patrol al-location history and corresponding crime report, to learn a richerDynamic Bayesian Network (DBN) model [28] of the interactionbetween the police officers and opportunistic criminals. The opti-mal patrol strategy is generated using the learned parameters of theDBN. While this approach predicts criminals’ behavior with highaccuracy for the problem in which the number of target areas issmall, it has three shortcomings: i) it cannot scale up to problemswith a large number of targets; ii) the algorithm performs poorlyin situations where the defender’s patrol data is limited; iii) theplanning algorithm only searches for a pure patrol strategy, whichquickly converges to a predictable pattern that can be easily ex-ploited by criminals.

In this paper, we focus on the problem of generating effectivepatrol strategies against opportunistic criminals in large scale ur-ban settings In order to utilize the superior performance of DBN ascompared to other models given ample data, we propose a novel ab-straction framework. This abstraction framework is our first contri-bution. In this framework we merge the targets with similar prop-erties and extract a problem with a small number of targets. Wecall this new problem the abstract layer and the original problemthe original layer. We first learn in the abstract layer using theDBN approach [28] and generate the optimal patrol strategy, thenwe propagate the learned parameters to the original layer and usethe resource allocation in the abstract layer to generate a detailedstrategy in the original layer. By solving the problem hierarchically

through multiple abstractions, we can generate the optimal strategyfor the original scenario.

Our second contribution is a layer generating algorithm, forwhich (i) we model it as a districting problem and propose a MILPin order to merge targets in the original problem into geographicallycompact and contiguous aggregated targets keeping the similarity(defined later) within them as homogeneous as possible; (ii) we de-velop a heuristic to solve this problem in large scale instances; (iii)we propose two approaches to find the optimal aggregated targets.Our third contribution is a planning algorithm that generates an op-timal mixed strategy against opportunistic criminals. We considera mixed strategy because (i) it broadens the scope of the defender’sstrategies; (ii) previous pure strategies depended on the model get-ting updated periodically; as mentioned earlier, the model usuallyconverged to a single pure strategy that is easy to exploit.

When the defender’s patrol data is limited or even missing inthe original layer, the learning approach in [28] overfits the data.Therefore, in order to solve this problem, we propose our fourthcontribution which is a heuristic model to propagate important fea-tures from the abstract layer to the original layer. We use modelsfrom behavioral game theory, such as Quantal Response, to extractthese features. In particular, we first approximate the learned DBNparameters in the abstract layer using behavioral parameters. Thenthe behavioral parameters are propagated to the original layer.

Finally, we evaluate our abstract game in two scenarios: the Uni-versity of Southern California (USC) campus [28] and Nashville,TN. We obtain data in USC from [28]. Data in Nashville, TN isobtained as part of the collaboration with the local police depart-ment.

2. RELATED WORKThere are five threads of research that are related to our problem.

The first line of work we compare with is game theoretic models,such as Stackelberg Security Games (SSG) [26], Opportunistic Se-curity Games (OSG) [27], Patrolling security games (PSG) [5] andPursuit Evasion Games (PEG) [17]. The interaction between po-lice and criminals is modeled as a Security Game. While SSG issuccessfully applied in security domains to generate randomizedpatrol strategies., e.g., in counter-terrorism and fare evasion checkson trains [18], it assumes that attackers are perfectly rational. Alot of recent research has focused on attackers with bounded ratio-nality. An example of such work is Opportunistic Security Games(OSG) [27]. In OSGs, attackers are opportunistic criminals whoare boundedly rational in planning the attacks but more flexible inexecuting the plan. An optimal patrol strategy against such oppor-tunistic adversaries is generated in OSGs. Recent work in leader-follower games, PSG, also has made progress in generating patrolstrategies against adversaries in arbitrary topology [4]. Differenttypes of adversaries in this game are considered in [6] while dif-ferent security resources are considered in [3]. Another example ofa game theoretic model is PEG, which models a pursuer attempt-ing to catch an evader [17]. In PEG, the evader is trying to avoidcapture and the pursuer is trying to capture the evader. However,as stated before, the adversary models in these games are hypoth-esized based on expert input and not detailed models — detailedin locations and time — learned from large amounts of real-worlddata that leads to the scale-up challenges addressed in our work.

The second area of work we compare with is data mining andmachine learning in the criminology domain. Recent research usesreal world crime data to analyze criminal behavior and recommendpatrol strategies for police. In [10], the author summarizes the gen-eral framework in this domain. In [21], crime detection and crimepattern clustering is achieved through data mining; in [12], machine

learning is used for criminal career analysis. However, in this areaof research, only crime data is considered. It does not explicitlymodel and learn the interaction between police and these criminalsfrom real world data; and nor does this work focus on planningpolice mixed strategies.

The third area of work we compare with is machine learning ingame theory. In [28], the interaction between a criminal and the de-fender is modeled as a Dynamic Bayesian Network (DBN). Crimeand patrol data are used to learn such interaction in this DBN andthe defender’s optimal strategy is generated. Unfortunately, thisapproach only works in small scale problems. When the numberof targets increases, the time complexity and the number of un-known variables increase dramatically; we show in Section 6, thatthis approach fails to run when the number of targets increases be-yond 20. In [8], the payoffs of attackers in SSGs are learned fromtheir responses against the defender’s strategy. However, in thisapproach, the goal is to show defender strategies that enable fastlearning of the adversary’s payoff instead of learning these modelsfrom existing detailed data of defender-adversary interactions. An-other example of such work is Green Security Games (GSG)[13,22], where poaching data may be used to learn a model of poach-ers’ boundedly rational decision making; as noted in our earlierpaper[28], our work complements theirs, and applying abstractionhierarchies introduced in this paper in GSGs remains an interestingissue for future work.

The last thread of recent research we compare with is the abstractgame that is widely used in large incomplete information gamessuch as Texas Hold’em [14, 16]. There are a number of differ-ent approaches including both lossless abstractions [15] and lossyabstractions [23]. In [11] and [1], sub-games are generated to cal-culate the Nash equilibrium in a normal form games. Abstractionshave also been brought into security games. In [2], abstraction isused to design scalable algorithms in PSGs. However, these worksfocus on clustering similar actions, strategies or states to formulatea simpler game. In our situation, we are physically merging thesimilar targets to generate simpler games. The criteria of mergingtargets is different from that of merging actions, strategies or states.Our differing criteria and approach for merging targets, differentmeans of propagating results of our abstractions, and our learningfrom real-world crime data set our work apart from this work.

3. PROBLEM STATEMENT

Figure 1: Sample Crime Report Figure 2: Sample Schedule

In this paper, we focus on limiting opportunistic crimes in largescale urban areas. Such large scale areas are usually divided intoN targets by the defenders. At the same time, defenders dividethe time into patrol shifts. T denotes the total number of shifts.At the beginning of each patrol shift, the defender assigns eachavailable patrol officer to a target and the officer patrols this targetin this shift. The criminals observe the defender’s allocation andseek crime opportunities by deciding the target to visit. In order tolearn the criminal’s opportunistic reaction to the defender’s alloca-tion, two categories of data are required for T shifts. The first isabout crime activity which contain crime details. Figure 1 showsa snapshot of this kind of data in a campus region. In this paper,we only consider the time and location information of crimes, ig-

noring the difference among different types of crimes. Therefore,we can summarize the crime report into a table like Table 1. Inthis table, columns represents the index of each target while rowsrepresents total number of shifts, 1...T. Each element in the tablerepresents the number of crimes at the corresponding target in thatshift. N × T data points are recorded in the table.

Shift 1 2 3 ... N1 1 1 2 ... 22 1 1 1 ... 13 2 1 1 ... 1

Table 1: Crime data

The second category is the pa-trol allocation schedule at theseshifts. The snapshot of suchdata is shown in Figure 2. Weignore the individual differencebetween officers and assumethat the officers are homoge-neous and have the same effect

on criminals’ behavior. Therefore, only the number of officers ateach target and shift affects criminals’ behavior and we can sum-marize the patrol data in the similar manner as crime reports, whichis shown in Table 2.

Shift 1 2 3 ... N1 2 1 1 ... 12 1 1 2 ... 23 2 1 1 ... 1

Table 2: Patrol data

Given the available data forcrime and patrol officers, ourgoal is to recommend efficientpatrol strategies to prevent op-portunistic crimes in problemswith a large number of targets.To begin with, we learn crimi-nals’ behavior from data and ap-

ply the abstract game framework to hierarchically learn the crimi-nal’s behavior. Next, we propose a planning algorithm that gener-ates the mixed strategy that optimizes the defender’s utility againstthe learned behavior of criminals.

4. ABSTRACT GAMEEven though previous approaches [28] deal with opportunistic

crimes, they cannot directly be applied to large scale problems.There are two reasons. First, over-fitting is inevitable in the learn-ing process of large scale problems. The number of unknown vari-ables in the learning process is O(N2) while the number of datapoints is O(N × T ) [28]. When N increases, the number of vari-ables gets close to the number of data points and causes over-fitting.The second reason is the runtime. The complexity of previous ap-proaches is at least O(NC+1T ) where C is the largest value thatany variable in the model of [28] can take and it grows quickly withN . In fact, our experiments shows that the algorithm does not con-verge in one day even with N = 25. Therefore, we propose theabstract game framework to deal with opportunistic crimes in largescale urban areas.

Figure 3: Game Ab-straction

The idea of abstracting the most es-sential properties of a complex realproblem to form a simple approxi-mate problem has been widely used inthe poker game domain [14]. Usingsuch an abstraction the problem can besolved hierarchically and a useful ap-proximation of an optimal strategy forthe real problem is provided. In this pa-per, we use the concept of abstractionto transform the large scale urban areaproblem into a smaller abstract prob-lem and solve it hierarchically. Figure3 illustrates the four steps in our gameabstraction framework. First, we needto generate the abstract layer from theoriginal layer (Section 4.1). Targets

that have similar properties are mergedtogether into aggregated targets. The set of aggregated targets iscalled the abstract layer while the set of original targets is calledthe original layer. Currently we only consider two layers: the orig-inal layer and the abstract layer. If the problem in the abstract layeris still too large to solve, we need to do further abstraction, whichwe will discuss in Section 4.5. After we obtain the abstract layer,the second step is to learn the criminal’s behavior and generate anoptimal patrol strategy in the abstract layer (Section 4.2). The thirdstep is to propagate information, such as criminal behavior features,from the abstract layer to the original layer (Section 4.3). Finally,we use the information from the abstract layer and data in the orig-inal layer to learn the criminal’s behavior and generate an optimalpatrol strategy in the original layer (Section 4.4).

4.1 Layer Generating AlgorithmWe model the layer generation as a Districting Problem [19, 9].

The districting problem is the well known problem of dividing ageographical region into balanced subregions with the notion ofbalance differing for different applications. For example, policedistricting problems focus on workload equality [9]. Our layer gen-eration is a districting problem that group targets in the originallayer into aggregated targets in the abstract layer. However, dis-tinct from the classic Districting problem where the resources arebalanced among different aggregated targets, in our problem, wetry to maximize the similarity of the targets inside the same aggre-gated target. We do so by modeling the similarity of targets withineach aggregated target and use this similarity measure as one of thecriteria in the optimization formulation of our problem.

When generating the aggregated targets, there are three princi-ples to follow. First, the aggregated targets should follow the ge-ometric constraints in the districting problem such as contiguity,compactness and environmental constraints. Contiguity means thatevery target is geographically connected; compactness means thatall targets in an aggregated target should be close together; andenvironmental constraints are the constraints for defender’s patrolconvenience. For example, if two neighboring targets are dividedby a highway, they should not be merged together. Second, thedissimilarity within the aggregated targets should be minimized.We consider two properties of target i, the number of crimes pershift with the defender’s presence ci1 and the number without thedefender’s presence ci0. For target i and target j, we define theDissimilarity distance function as Disij = |ci1 − cj1|+ |ci0 − c

j0|.

Third, the algorithm should consider the scalability constraintfor learning algorithm. Let N denote the number of targets in theoriginal layer and n denote the largest scale of problem that thelearning and planning algorithms can scale up to. Then there canbe no more than n targets inside each aggregated target and nomore than n aggregated targets in the abstract layer. Therefore,N ≤ n2 in the original layer. When N > n2, we need multiplelayer abstraction that will be introduced later. As we prove next inLemma 1, the more the aggregated targets are in the abstract layer,the less information is lost during the abstraction. Hence, we wouldwant to have as many targets as possible in the abstract layer. Thus,we set n aggregated targets in the abstract layer.

Let I = {1, . . . , N} be a set of targets in the original layer.A partition of size K of this set I is a collection of sets {Ik}Kk=1

such that Ik 6= ∅ for all k ∈ {1, . . . ,K}, Ik ∩ Il = ∅ for allk, l ∈ {1, . . . ,K}, k 6= l and

⋃Kk=1 Ik = I . {Ik}Kk=1 is the

set of the aggregated targets in the abstract layer. Let PK(I) de-note the set of all partitions of I of size K. Given Ik ⊂ I wedefine its inner Dissimilarity as Dis(Ik) =

∑i,j∈Ik

Disij =∑i,j∈Ik

|ci1−cj1|+|ci0−cj0| . Also we define its Inertia as In(Ik) =

minj∑i∈Ik

dij , with dij denoting the physical distance betweenthe geometric centers of targets i, j. In our districting process wewant to find a partition which achieves both low inner Dissimilar-ity and Inertia over all elements of the partition. Given α > 0 asa normalization parameter, we define the information loss functionLI(K) as the lowest cost with a partition of sizeK, mathematicallyLI(K) = min{Ik}Kk=1

∈PK(I)

∑Kk=1 αIn(Ik) +Dis(Ik).

LEMMA 1. The information loss decreases with K, that isLI(K + 1) ≤ LI(K).

The proof of Lemma 1 is in the appendix (http://bit.ly/1ND8liH) . Based on these three principles, we propose a mixedinteger linear program (MILP) to solve the districting problem.We apply an extension of the capacitated K-median problem withK = n. While the capacitatedK-median problem [24] satisfies thescalability constraint by setting a maximum capacity for each ag-gregated target, it cannot handle the geometric constraints such ascontiguity. A counterexample is shown in the appendix. In this pa-per, we handle the geometric constraints by considering the inertiaof each aggregated target as part of the information loss function.

minx,y,z α∑i,j dijyij +

∑ik zik

s.t.∑j yij = 1 ∀i ∈ I

yij ≤ xj ∀i, j ∈ I∑j xj = n∑j yij ≤ n ∀j ∈ I

zik ≥ Disik(yij + ykj − 1) ∀i, k, j ∈ Izik ≥ 0 ∀i, k ∈ Iyij + ykj ≤ 1 ∀j ∈ Iyij , xj ∈ {0, 1} ∀i, j ∈ I

(1)

xj is a binary variable. It is 1 if the target j is the center of anaggregated target and 0 otherwise. The variable yij takes the value1 when the target i is allocated to the aggregated target centered inj and 0 otherwise. The variable zik is a continuous non-negativevariable that takes the value Disik when target i and target k areallocated to the same aggregated target, otherwise zik is 0. The ob-jective function is the weighted sum of inertia and dissimilarity. αrepresents the trade-off between geometric shape and the similaritywithin each aggregated target.

The first set of constraints ensures that every target is allocatedto an aggregated target. The second set of constraints ensures thatthe center of an aggregated target belongs to this aggregated target.The third expression states that there are n aggregated targets. Thefourth set of inequalities ensures the size of every aggregated targetto be no greater than n. The fifth and sixth constraint ensures thatzik will take the value Disik when target i and target k are allo-cated to the same aggregated target, otherwise zik will be 0. Theseventh constraint is an example of environmental constraints thattarget i and target k cannot be in the same aggregated target.

Directly solving this MILP is NP-hard [20]. Therefore we usethe heuristic constraint generation algorithm (Algorithm 1) to ap-proximately solve the problem. The algorithm has two phases: first,the location problem is solved as a K-median problem. In the sec-ond phase, we use the constraint generation technique [7] to solvethe optimization problem. The iterative constraint generation al-gorithm is shown as the for loop (line 2-9). To start with, all theconstraints zik ≥ Disik(yij + ykj − 1) for i, j, k are removedcompletely (denoted by the empty set Cuts in line 1), and then ineach iteration of the for loop the MILP is solved (line 3) and then

Algorithm 1 Constraint Generation Algorithm (I,K)1: Center← Location_Problem(I,K); Cuts=∅2: for i = 1, · · · ,MAX_IT do3: y∗, z∗ ← Allocation_Phase(Center, Cuts)4: i∗, j∗, k∗ = argmini,j,k z

∗ik −Disik(yij + ykj − 1)

5: if (z∗i∗k∗ −Disi∗k∗(yi∗j∗ + yk∗j∗ − 1)) ≥ 0 then6: break7: else8: Cuts← Cuts ∪ {zi∗k∗ ≥ Disi∗k∗(yi∗j∗ + yk∗j∗ − 1)}9: end if

10: end for11: return {y∗}, objective_function

we check whether any of the left out constraints are violated (line4, 5). If yes, then the most violated constraint is added to Cutsor else the loop stops. The maximum number of iterations is lim-ited by MAX_IT . Constraint generation guarantees an optimalsolution given large enough MAX_IT .

4.2 Abstract Layer

Figure 4: DBNframework

Learning Algorithms: As noted ear-lier, having generated the abstract layer,the next step is learn the adversary modelat the abstract layer. As stated before, theDynamic Bayes Network (DBN) learningalgorithm presented in [28] could not beused in the original layer due to scalingdifficulties; however, with a sufficientlysmall number of targets in the abstractlayer, we can now use it. To illustrateits operation, we reproduce the operationwith N targets as shown in Figure 4.Three types of variables are considered inthe DBN: squares in the top represent thenumber of defenders at aggregated target iduring shift t, Di,t, squares in the bottomrepresent the number of crimes at aggre-gated target i during shift t, Yi,t, whilecircles represents the number of crimi-nals at aggregated target i during shift t,Xi,t. As shown in Figure 4, there are

two transitions in the DBN: the criminal’s transition from shift tto t + 1, which is modeled as the transition probability and thecrime transition at shift t, which is modeled as the crime out-put probability. Mathematically, a transition probability is definedas P (Xi,t+1|D1,t, ..., DN,t, X1,t, ..., XN,t) and the crime outputprobability is defined as P (Yi,t|D1,t, ..., DN,t, X1,t, ..., XN,t).This model uses two matrices to represent the transition probabil-ities, the movement matrix A which consists of all the criminal’stransition probability P (Xi,t+1|D1,t, ..., DN,t, X1,t, ..., XN,t)and the crime matrixB which consists of all the crime output prob-ability P (Yi,t|D1,t, ..., DN,t, X1,t, ..., XN,t). A and B containsCN × CN × CN unknown parameters.

Given available data about Di,t (patrol schedule), Yi,t (crimereport), this model applies the Expectation Maximization algorithmto learn A and B while estimating Xi,t. The detail of this learningmodel is present in [28]. The novelty in this paper is propagatingadversary behavior parameters (A andB) from the abstract layer tothe original layer, which we discuss in Section 4.3; but we do that,we discuss planning in the abstract layer.

Planning Algorithms: In this paper, we focus on planning withmixed strategies for the defender rather than the pure strategy plans

from previous work [28]. This change in focus is based on two keyreasons. First, this change essentially broadens the scope of the de-fender’s strategies; if pure strategies are superior our new algorithmwill settle on those (but it tends to result in mixed strategies). Sec-ond, previous work [28] on planning with pure strategies dependedon repeatedly cycling through the following steps: planning multi-ple shifts of police allocation for a finite horizon, followed by up-dating of the model with data. This approach critically depended onthe model getting updated periodically in deployment. Such peri-odic updating was not always easy to ensure. Thus, within any onecycle, the algorithm in [28] led to a single pure strategy (single po-lice allocation) being repeated over the finite horizon in real-worldtests as it tried to act based on the model learned from past data;such repetition was due to a lack of updating of the criminal modelwith data, and in the real-world, the criminals would be able to ex-ploit such repetition. Instead, here we plan for a mixed strategy.We assume that the model updates may not occur frequently and asa result we plan for a steady state.

We model the planning procedure as an optimization problemwhere our objective is to maximize the defender’s utility per shift.After the defenders’ (mixed) strategy is deployed for a long time,criminals receive perfect information of the strategy and their(probabilistic) reaction will not change over time. As a result, thecriminals’ distribution becomes stationary and this is called crimi-nals’ stationary state. In our case, ergodicity guarantees unique sta-tionary state (see appendix). Our planning algorithm assumes crim-inals’ stationary state when maximizing the defender’s utility. Wedefine defender’s utility as the negation of the number of crimes.Therefore, the objective is to minimize the number of crimes thathappen per shift in the stationary state. Let’s define I = {i} asthe set of aggregated targets, D as the total number of defendersthat are available for allocation; dI = {di} as the set of defender’sallocation at target set I , xI = {xi} as the set of criminal’s station-ary distribution at target set I with respect to defender’s strategy dIand yI = {yi} as the set of expected number of crimes at targetI . Note that C is the largest value that the variables Di, Xi and Yican take. The optimization problem can be formed as follows:

minimizedI

∑i∈I yi

subject to 0 ≤ xi ≤ C, i ∈ I,0 ≤ di ≤ C, i ∈ I,∑

i∈I di ≤ D,yi =

∑Yi,t

Yi,t·

P (Yi,t|d1, ..., dN , x1, ..., xN ), i ∈ I,xi =

∑Xi,t+1

Xi,t+1·

P (Xi,t+1|d1, ..., dN , x1, ..., xN ), i ∈ I.

(2)

In this optimization problem, we are trying to minimize the totalnumber of crimes occurring in one shift while satisfying five setsof constraints. The first two constraints ensure the defender andcriminal’s distribution are non-negative and no more than an upperbound C. The third constraint represents the constraint that thenumber of deployed defender resources cannot be more than theavailable defender resources. The fourth constraint is the crimeconstraint. It sets yi to be the expected number of crime at targeti. The last constraint is the stationary constraint, which means thatthe criminals’ distribution is not changing from shift to shift withrespect to the patrol strategy dI . The transitions are calculated bymovement matrix A and crime matrix B. The details of the crimeand stationary constraint are shown in the appendix.

4.3 Propagation of learned criminal modelIn the previous section, we generate the patrol allocation for the

aggregated targets in the abstract layer. In order to provide pa-trolling instructions for the original layer, we propagate the learnedcriminal model in the abstract layer to the original layer. We need toaddress two cases: when there is no detailed patrol data and whenthere is. In particular, we have found that some police departmentsrecord the location of police patrols in detail at the level of targetsin the original layer, but many others specifically only keep approx-imate information and do not record details (even if they record allcrime locations in detail); thus leading to the two cases. We start bydescribing the case with sufficient patrol data in the original layer.

Direct learning (sufficient data): When there is detailed patroldata in the original layer and nothing is approximated away, weknow the numbers of police at each target in the original layer ateach shift. Then, we can directly learn A and B in this DBN. Thelearning algorithm is same as that applied in the abstract layer. Thedata used in the algorithm is the crime report and patrol scheduleinside each aggregated target. While we directly learn A and B,computation of the patrol strategy at the abstract layer affects thepatrol strategy in the detailed layer as discussed in Section 4.4.

Parameter Propagation (limited data): If the patrol data inthe original layer is limited, the DBN model that we learned in theoriginal layer will be inaccurate if we still apply the same learn-ing algorithm as the abstract layer to learn matrices A and B. Oneremedy is to provide addition criminal information to the originallayer from the abstract layer to help the process of learning criminalmodel in the original layer. However, in the abstract layer, move-ment matrix A and crime matrix B represent the criminal’s behav-ior in aggregated targets. It cannot directly describe the criminal’sbehavior in the targets in the original layer. Therefore, we proposea human behavior based model of extracting behavior parametersfrom A and B in the abstract layer. Then we set these behavior pa-rameters of an aggregated target as the behavior parameters for thetargets contained within this aggregated target in the original layer.

Parameter extraction: We introduce the process of using a hu-man behavior model to extract the behavior parameters fromA andB. The basic assumption of a human behavior model is that thecriminal follows certain patterns when moving from shift to shift.Specifically, the criminals follow the movement by the well es-tablished Quantal Response (QR). In the learning algorithm [28],one simplification made was breaking down the criminals’ transi-tion probabilities into marginal probability P (Xj,t+1|Di,t, Xi,t)which represents the movement of a criminal from target i to tar-get j. Based on the Quantal Response model, we approximatethis movement using the following equation: P (Xj,t+1 = 1) =

eAttj∑

n∈N eAttnwhere Attn is the attractiveness property of target

n. In the DBN, the movement depends not only on the attrac-tiveness, but also on the allocation of defenders and criminals atprevious shift. Therefore, we formulate P (Xj,t+1 = 1|Di,t, Xi,t)as (λi, µi ≥ 0):

eAttj∑

n∈N eAttn· eλiXi,t+µiDi,t , if i 6= j

eAttj∑

n∈N eAttn· eλiXi,t−µiDi,t , otherwise

(3)

The reason for the above effect of defender is that the defender attarget i disperses criminals to other targets. However, λ, µ and Attare not known and we need to learn them from data. Our approachto compute λ, µ and Att is to find their values that minimize theL1 distance between P (Xj,t+1 = 1|Di,t, Xi,t) and the learnedmarginal probability P (Xj,t+1 = 1|Di,t, Xi,t). We can formulatethis problem as the following optimization:

minAtt,λ,µ∑i,j,Di,t,Xi,t

||P (Xj,t+1 = 1|Di,t, Xi,t)−

P (Xj,t+1 = 1|Di,t, Xi,t)||subject to µi ≥ 0, λi ≥ 0, i = 1, ..., N

The constraints represent the positive effect of number of crimi-nals on the transition probability and more defenders lead to fasterdispersion of criminals. λ, µ and Att are the behavior parametersthat we propagate to original layer.

Since λ and µ represent the influence of the number of criminalsand number of defenders on the criminals’ movement in the aggre-gated target, it is reasonable to assume that the criminals’ move-ment in the targets that belong to the aggregated target inherit theseparameters. In other words, this means that the influence of thenumber of criminals and defenders is the same within the aggre-gated target. At the same time, Att measures the availability of thecrime opportunities. Therefore, within one aggregated target, theattractiveness is distributed among the targets proportional to thetotal number of crimes in each target. For example, if the attrac-tiveness of an aggregated target I (made up of I1 and I2) is 0.6, thetotal number of crimes at target I1 is 80 while that at target I2 is40, then the attractiveness of A1 is 0.4 while that of A2 is 0.2. λ,µ and Att for each target are the behavior parameters that will beused in crime and stationary constraints in the planning algorithm.

4.4 Computing Strategy in the Original LayerIn the previous section, we generated the adversary behavior pa-

rameters in the original layer. In order to provide patrolling instruc-tions for the original layer, we utilize the strategy in the abstractlayer to assign resources in the original layer. Then, combinedwith the propagated adversary behavior parameters we generate thestrategy at the original layer.

Resource Allocation: In the abstract layer, the optimal strategyrecommends the number of resources allocated to each aggregatedtarget. We use this recommendation as a constraint on the numberof resources in planning within the aggregated targets at the orig-inal layer. For example, the abstract layer may provide 0.8 as theallocation to an aggregated target say X; then we plan patrols in Xin the original layer using 0.8 as the total number of resources.

Next, in the original layer, we treat each aggregated target in theabstract layer as an independent DBN as shown in Figure 4. Thesame algorithm for generating a mixed strategy in the abstract layercan be applied in each of the independent DBNs. The optimizationproblem is the same as Equation 2. D is the total number of re-sources allocated to these aggregated targets (e.g., 0.8 to target X).

In addition, the formulations of crime and stationary constraintsrequired in the computation of the mixed strategy are different forthe scenario with sufficient and limited data. For the scenario withsufficient data these constraints are formulated using the parametersA and B of the DBN that is learned in this original layer. Forthe scenario with limited data the propagated values of λ, µ andAtt are used to estimate the the A and B parameters for the DBNrepresentation of the adversary behavior in the original layer. Theestimation is the inversion of parameter extraction, and it happensin the original layer. For example, we use Equation 3 to estimatethe parameters using λ, µ and Att. The details are presented in theappendix. Then, these reconstructedA andB are used to formulatethe crime and stationary constraints.

4.5 Extended Abstract GameWhen n2 < N , we can use two layers of abstraction to solve

the problem. However, when the real problem has N > n2 tar-

gets, even two layered abstraction does not suffice since there mustbe a layer in the game with more than n targets. Therefore, wepropose the multiple layer framework to handle problems with anarbitrarily large number of targets. This framework is an extensionof the two layer abstract game. We apply an iterative four step pro-cess. As a first step, we need to decide the number of layers aswell as the districting of targets for each of the layers. Consider-ing the scalability constraints (recall that there cannot be more thann targets within each aggregated target), the number of layers isM = blognNc + 1. We denote the original layer as Layer 1 andthe layer directly generated from Layer m as layer m + 1. In thisnotation, the topmost abstract layer is Layer M . The second stepis learning criminals behavior in the top layer. The third step is togenerate a patrol strategy at this layer. The fourth step is to propa-gate parameters to the next layer. We keep executing steps two tofour for each layer until we reach the original layer. At each layer,we decide whether to do parameter propagation based on the avail-ability of the patrol data. If we have sufficient patrol data at layerm, we do direct learning at layer m. Otherwise, we do parameterpropagation from layer m+ 1 to layer m.

We propose three different layer generation algorithms. The firstalgorithm is the direct algorithm. For example, if N = 50 andn = 5. Then, there should be M = 3 layers. For layer 1, therewill be 50 targets. For layer 2, the number of targets could be anyinteger between 10 to 25. For layer 3, the number of targets can be2 to 5. The direct learning tries all the combinations of three layersand runs the MILP for each combination to generate the optimalsegmentation. It calls the MILP in Section 4.1 for O(NM · M)times; the second algorithm is a dynamic programming approachthat ensures the solution is globally optimal. The MILP is calledO(N2 ·M) times; the third algorithm is the greedy algorithm thatsets the number of targets to be maximum, which for the mth layeris nM+1−m. The number of calls is M while the solution is notnecessarily optimal. Details are in the appendix.

5. REAL WORLD VALIDATION

Figure 5: Campus map 1 Figure 6: Campus map 2

We use two sets of real world data to validate the game abstrac-tion framework. In the first case we use the data from the Universityof Southern California (USC) campus that is provided by [28]. Wethank the authors for providing three years (2012-2014) of crimereport and patrol schedule from the USC campus. The number oftotal crime events is on the order of 102. [28] reports that the cam-pus patrol area (USC campus and its surroundings) is divided intofive patrol areas, which are shown in Fig 5. In order to make thepatrols more efficient, the police officers wish to further divide thewhole campus into 25 patrol areas and get patrol recommendationson these 25 patrol areas. There are two tasks for us, (a) startingfrom city blocks (there are 298 city blocks and they form the basisof the USC map), create 25 separate "targets", as in our layer gen-eration problem; (b) generate an optimal patrol strategy for these25 targets. The creation of 25 targets is also a districting problemand the technique in Section 4.1 can be directly applied. The 25targets generated by the districting algorithm is shown in Figure 6.

We treat these 25 targets as the original layer. n is set to be 5 as theruntime of learning and planning algorithm with n = 5 is reason-ably small. So then we use two layer game abstraction to solve thisproblem with 25 targets. The abstract layer is the five patrol areasin Fig 5. This is because of the center area (the darkest area) is thecampus itself and is separated from its environment by fences andgates. These environmental constraints cause our layer generationto automatically create the area into 5 targets as shown in Figure 5.Additionally, police only record their presence in the five areas, andthus, we do not have detailed police presence data; as a result, weuse our behavior learning to propagate parameters from the abstractlayer to the original layer.

Figure 7: Citymap

In the second case, we use data aboutcrime and detailed police patrol locationsin Nashville, TN, USA. The data covers atotal area of 526 sq. miles. Only burglar-ies (burglary/breaking and entering) havebeen considered for the analysis. Burglaryis the chosen crime type as it is a majorportion of all property crimes and is welldistributed throughout the county. Data for10 months in 2009 is used. The number oftotal crime events is on the order of 103.Observations that lacked coordinates were

geocoded from their addresses. Police presence is calculated fromGPS dispatches made by police patrol vehicles. Each dispatch con-sists of a unique vehicle identifier, a timestamp and the exact loca-tion of the vehicle at that point in time. We divide the whole cityinto N = 900 targets as shown in Figure 7. Since n is 5, the num-ber of layers we need is M = blog5 900c + 1 = 5. We use themultiple layer abstraction framework to solve this problem.

6. EXPERIMENTAL RESULTSExperiment setup. We use MATLAB to solve our optimiza-

tion problems. There are two threads of experiments, one on theUSC campus problem and the other on Nashville, TN problem. Toavoid leaking confidential information of police departments, allcrime numbers shown in the results are normalized. The experi-ments were run on a machine with 2.4 GHz and 16 GB RAM.

Game Abstraction Framework: Our first experiment is oncomparing the performance of our game abstraction frameworkwith the DBN framework proposed in [28] for large scale prob-lems. Since the DBN framework cannot even scale to problemswith 25 targets, in this experiment we run on problems with sub-sets containing N targets (5 ≤ N < 25) out of these 25 targets inthe USC campus. As shown in Figure 8, we compare the runtimeof these two frameworks. The x-axis in Fig. 8 is the number of tar-getsN in the problem. For eachN , we try ten different subsets andthe average runtime is reported. The y-axis indicates the runtime inseconds. The cut-off time is 3600s. As can be seen in Figure 8, theruntime of the DBN framework grows exponentially with the scaleof the problem and cannot finish in an hour when N = 20. At thesame time, the runtime of the game abstraction framework growslinearly with the scale of the problem. It takes less than 5 minutesto solve the problems with N = 20. This indicates that the DBNframework fails to scale up to large scale problems while the gameabstraction framework can handle many more targets.

In Figure 9 we compare the prediction accuracy of these twodifferent frameworks. We divide the 36 months’ data sets into twoparts, the first 35 months’ data is used for learning while we predictthe crime distribution for the last month and compare it with thereal crime data in that month. For every target and every shift, wemeasure the prediction accuracy as the predicted probability of the

number of crimes reported in the data for that target and shift. Forexample, for target i and shift t, our prediction is that there is 30%probability that no crime occurs and 70% that one crime occurswhile in the data there is one crime at target i in shift t. Then, theprediction accuracy for target i for shift t is 0.7. The reported ac-curacy is the average accuracy over all targets and all shifts over allten different subsets. The higher the accuracy, the better our predic-tion. As can be seen in Figure 9, the game abstraction frameworkachieves similar prediction accuracy compared to the DBN algo-rithms given any number of targets in the problem. This indicatesthat even through information may be lost during the abstraction,the game abstraction framework captures important features of thecriminal and performs as well as the exact DBN framework whilerunning 100 of times faster.

Layer Generation Algorithm: Next, we use the data from thecity to evaluate the performance of our layer generation algorithms.Again, we run the layer generation algorithms on problems withsubsets containing N targets (N ≤ 900) out of the 900 targets inthe city map. For each N , we try ten different subsets and reportthe average value except when N = 900 for which only one sub-set is possible. Figure 10 compares the runtime of different layergeneration algorithms in log format. Three different algorithms arecompared, the direct algorithm (Direct) that traverses all possiblelayer combinations; the dynamic programming algorithm (DP) andthe greedy algorithm (Greedy). The x-axis in Fig. 10 is the numberof targets N . For N = 25, two layers are needed; for N = 50,three layers are needed; for N = 200, four layers are needed andfor N = 900, five layers are needed. The y-axis is the runtime ofdifferent algorithms in seconds. The cut-off time is set at 36000s.When N = 25, the runtime of these three algorithms are the samebecause the layer generation is unique. The number of targets inlayer 2 is 5. When N = 50, the runtime of the direct algorithmis the same as that of the DP algorithm while the runtime of thegreedy algorithm is significantly lower. When N = 200, the directalgorithm cannot finish in 10 hours; the DP algorithm takes aroundfive hours while greedy algorithm finishes in less than 10 minutes.When N = 900, both direct learning and DP are cut off whilethe runtime for greedy is less than 15 minutes. This validates ourtheoretical result that the runtime of direct algorithm grows expo-nentially with the scale of the problem, that of DP grows polyno-mially and that of greedy algorithm grows linearly with the numberof layers. Since both direct and DP algorithm cannot scale up to theproblem with N = 900, we use the greedy algorithm as the layergeneration algorithm in the city problem.

In Figure 11, we compare the information loss of different layergeneration algorithms. The information loss is defined as the objec-tive in Equation 1. As can be seen in Fig. 11, the information lossof DP is the same as that of direct learning in any situations. This isbecause DP ensures a globally optimal solution. At the same time,the information loss of the greedy algorithm is higher than that ofthe DP algorithm but no more than 15% higher. This indicates thatwhile greedy algorithm cannot ensure global optimal informationloss, it can reach a good approximation in reasonable runtime.

Learning: Third, we evaluate the performance of our learningalgorithm. Game abstraction is used for both problems and we eval-uate the predictions in the original layer. The result shown in Figure12 and Figure 13 compares the prediction accuracy of different al-gorithms in USC campus and the city problem respectively. Threedifferent algorithms are compared: (1) the Random approach, inwhich the probabilities of each situation are the same (Random),(2) game abstraction with direct learning for both the abstract andoriginal layer (DL) and (3) game abstraction with parameter prop-agation in the original layer (PP). We divide the whole data sets

5 10 15 200

1000

2000

3000

Run

time/

s

DBNAbstract

Figure 8: Runtime

5 10 15 200

0.2

0.4

0.6

0.8

1

Pre

dict

ion

Acc

urac

y

DBNAbstract

Figure 9: Accuracy

25 50 200 9000

3

30

300

3000

30000

Number of targets

Run

time/

s

DirectDPGreedy

Figure 10: Runtime

25 50 200 9000

2

4

6

Number of targets

Info

Los

s

DirectDPGreedy

Figure 11: Information Loss

1 2 3 40

0.2

0.4

0.6

0.8

1

dataset

Pre

dict

ion

accu

racy

PPDLRandom

Figure 12: Accuracy (USC)

1 2 3 40

0.2

0.4

0.6

0.8

1

dataset

Pre

dict

ion

accu

racy

PPDLRandom

Figure 13: Accuracy (city)

1 2 3 40

0.2

0.4

0.6

0.8

1

dataset

Num

ber

of c

rimes

RealReal−EOptimal

Figure 14: Plan (USC)

1 2 3 40

0.2

0.4

0.6

0.8

1

dataset

Num

ber

of c

rimes

RealReal−EOptimal

Figure 15: Plan (city)

into four equal parts. For each part, the first 90% of data is usedfor training while we test on the last 10% of data. The x-axis inFig. 12 and 13 is the index of the part of data that we evaluate on.y-axis indicates the prediction accuracy on the test set. As can beseen in both figures, the accuracy of both game abstraction basedapproaches are higher than that of the baseline random algorithmin all the test sets. This indicates that game abstraction models helpimprove the prediction in large scale problems. In addition, param-eter propagation at the original layer outperforms direct learningat this layer in the USC problem in Figure 12. Direct learning out-performs parameter propagation in Nashville problem in Figure 13.This is because the patrol data at the original layer in USC is lim-ited. That is, only the aggregate number of police resources overseveral targets is available while the resources at each target remainunknown. Parameter propagation is better at handling limited pa-trol data. However, the patrol data is adequate in the city problemand direct learning is a better fit in such situations. Therefore, inthe planning section, we use parameter propagation as the learningalgorithm in the USC and direct learning as the learning algorithmin Nashville.

1 2 3 4 50

500

1000

1500

2000

Layer

Run

time/

s

Set1Set2Set3Set4

Figure 16: Runtime

Planning: Next, weevaluate the perfor-mance of our planningalgorithm in both theproblems. Figure 14 and15 compare strategiesgenerated using thegame abstraction frame-work with the actualdeployed allocationstrategy generated by thedomain experts. Threedifferent scenarios are compared: the real number of crimes,shown as Real; the expected number of crimes with manuallygenerated strategies and learned adversary model with gameabstraction, shown as Real-E and the expected number of crimeswith the optimal strategy computed using game abstraction, shownas Optimal. As shown in Figure 14 and 15, the expected number ofcrime with manually generated strategy is close to the real numberof crimes, which indicates game abstraction model captures thefeature of criminals and provide good estimation of the real

number of crimes. In addition, strategy generated using the gameabstraction is projected to outperform the manually generatedstrategy significantly. This shows the effectiveness of our proposedpatrol strategy as compared to the current patrol strategy.

Runtime: Finally, we break down the total runtime of the gameabstraction framework in the city problem layer by layer and showit in Figure 16. The x-axis is the index of the layer, which goesfrom the original layer (Layer 1) to the top layer (Layer 5). The y-axis is the total runtime of the propagation, learning and planningalgorithm in that layer. As can be seen, the runtime increases asthe layer index decreases except for Layer 1. This is because ingreedy layer generation, for the fifth layer the number of targets is5, and for the fourth layer it is 52, for third layer it is 53, for thesecond layer it is 54 but for the first layer it is only 900. Therefore,the number of targets within each aggregated target in layer two isless than 3 < n = 5. Therefore, the runtime in layer 1 is faster.However, the total runtime of the whole process is less than anhour in each data set. Therefore, the game abstraction frameworkcan be extended to large scale problems with reasonable runtimeperformance.

7. CONCLUSIONSThis paper introduces a novel game abstraction framework to

learn and plan against opportunistic criminals in large-scale urbanareas. First, we model the layer-generating process as a districtingproblem and propose a MILP based technique to solve the problem.Next, we propose a planning algorithm that outputs randomizedstrategies. Finally, we use a heuristic propagation model to handlethe problem with limited data. Experiments with real data in twourban settings shows that our framework can handle large scale ur-ban problems that previous state-of-the-art techniques fail to scaleup to. Further, our approach provides high crime prediction accu-racy and the strategy generated from our framework is projected tosignificantly reduce crime compared to current police strategy.

8. ACKNOWLEDGEMENTThis research is supported by MURI grant W911NF-11-1-0332

and Vanderbilt University Discovery grant.

REFERENCES[1] A. Basak and C. Kiekintveld. Abstraction using analysis of

subgames. In IJCAI Workshop on Algorithmic Game Theory,2015.

[2] N. Basilico and N. Gatti. Automated abstractions forpatrolling security games. In AAAI, 2011.

[3] N. Basilico and N. Gatti. Strategic guard placement foroptimal response toalarms in security games. In Proceedingsof the 2014 international conference on Autonomous agentsand multi-agent systems, pages 1481–1482. InternationalFoundation for Autonomous Agents and MultiagentSystems, 2014.

[4] N. Basilico, N. Gatti, and F. Amigoni. Leader-followerstrategies for robotic patrolling in environments witharbitrary topologies. In Proceedings of The 8th InternationalConference on Autonomous Agents and MultiagentSystems-Volume 1, pages 57–64. International Foundation forAutonomous Agents and Multiagent Systems, 2009.

[5] N. Basilico, N. Gatti, and F. Amigoni. Patrolling securitygames: Definition and algorithms for solving large instanceswith single patroller and single intruder. ArtificialIntelligence, 184:78–123, 2012.

[6] N. Basilico, N. Gatti, T. Rossi, S. Ceppi, and F. Amigoni.Extending algorithms for mobile robot patrolling in thepresence of adversaries to more realistic settings. InProceedings of the 2009 IEEE/WIC/ACM International JointConference on Web Intelligence and Intelligent AgentTechnology-Volume 02, pages 557–564. IEEE ComputerSociety, 2009.

[7] D. Bertsimas and J. N. Tsitsiklis. Introduction to linearoptimization, volume 6. Athena Scientific Belmont, MA,1997.

[8] A. Blum, N. Haghtalab, and A. D. Procaccia. Learningoptimal commitment to overcome insecurity. In Advances inNeural Information Processing Systems, pages 1826–1834,2014.

[9] V. Bucarey, F. Ordóñez, and E. Bassaletti. Shape and balancein police districting. In Applications of Location Analysis,pages 329–347. Springer, 2015.

[10] H. Chen, W. Chung, J. J. Xu, G. Wang, Y. Qin, and M. Chau.Crime data mining: a general framework and someexamples. Computer, 37(4):50–56, 2004.

[11] V. Conitzer and T. Sandholm. A technique for reducingnormal-form games to compute a nash equilibrium. InProceedings of the fifth international joint conference onAutonomous agents and multiagent systems, pages 537–544.ACM, 2006.

[12] J. S. De Bruin, T. K. Cocx, W. Kosters, J. F. Laros, J. N. Kok,et al. Data mining approaches to criminal career analysis. InData Mining, 2006. ICDM’06. Sixth InternationalConference on, pages 171–177. IEEE, 2006.

[13] F. Fang, P. Stone, and M. Tambe. When security games gogreen: Designing defender strategies to prevent poaching andillegal fishing. In International Joint Conference on ArtificialIntelligence (IJCAI), 2015.

[14] A. Gilpin and T. Sandholm. Better automated abstractiontechniques for imperfect information games, with applicationto texas hold’em poker. In Proceedings of the 6thinternational joint conference on Autonomous agents andmultiagent systems, page 192. ACM, 2007.

[15] A. Gilpin and T. Sandholm. Lossless abstraction of imperfect

information games. Journal of the ACM (JACM), 54(5):25,2007.

[16] A. Gilpin, T. Sandholm, and T. B. Sørensen. A heads-upno-limit texas hold’em poker player: discretized bettingmodels and automatically generated equilibrium-findingprograms. In Proceedings of the 7th international jointconference on Autonomous agents and multiagentsystems-Volume 2, pages 911–918. International Foundationfor Autonomous Agents and Multiagent Systems, 2008.

[17] J. P. Hespanha, M. Prandini, and S. Sastry. Probabilisticpursuit-evasion games: A one-step nash approach. InDecision and Control, 2000. Proceedings of the 39th IEEEConference on, volume 3, pages 2272–2277. IEEE, 2000.

[18] A. X. Jiang, Z. Yin, C. Zhang, M. Tambe, and S. Kraus.Game-theoretic randomization for security patrolling withdynamic execution uncertainty. In Proceedings of the 2013international conference on Autonomous agents andmulti-agent systems, pages 207–214. InternationalFoundation for Autonomous Agents and MultiagentSystems, 2013.

[19] J. Kalcsics, S. Nickel, and M. Schröder. Towards a unifiedterritorial design approachâATapplications, algorithms andgis integration. Top, 13(1):1–56, 2005.

[20] O. Kariv and S. L. Hakimi. An algorithmic approach tonetwork location problems. ii: The p-medians. SIAM Journalon Applied Mathematics, 37(3):539–560, 1979.

[21] S. V. Nath. Crime pattern detection using data mining. InWeb Intelligence and Intelligent Agent TechnologyWorkshops, 2006. WI-IAT 2006 Workshops. 2006IEEE/WIC/ACM International Conference on, pages 41–44.IEEE, 2006.

[22] T. H. Nguyen, F. M. Delle Fave, D. Kar, A. S.Lakshminarayanan, A. Yadav, M. Tambe, et al. Making themost of our regrets: Regret-based solutions to handle payoffuncertainty and elicitation in green security games. InDecision and Game Theory for Security, pages 170–191.Springer, 2015.

[23] T. Sandholm and S. Singh. Lossy stochastic game abstractionwith bounds. In Proceedings of the 13th ACM Conference onElectronic Commerce, pages 880–897. ACM, 2012.

[24] H. D. Sherali and F. L. Nordai. Np-hard, capacitated,balanced p-median problems on a chain graph with acontinuum of link demands. Mathematics of OperationsResearch, 13(1):32–49, 1988.

[25] M. B. Short, M. R. D’ORSOGNA, V. B. Pasour, G. E. Tita,P. J. Brantingham, A. L. Bertozzi, and L. B. Chayes. Astatistical model of criminal behavior. Mathematical Modelsand Methods in Applied Sciences, 18(supp01):1249–1267,2008.

[26] M. Tambe. Security and game theory: algorithms, deployedsystems, lessons learned. Cambridge University Press, 2011.

[27] C. Zhang, A. X. Jiang, M. B. Short, P. J. Brantingham, andM. Tambe. Defending against opportunistic criminals: Newgame-theoretic frameworks and algorithms. In Decision andGame Theory for Security, pages 3–22. Springer, 2014.

[28] C. Zhang, A. Sinha, and M. Tambe. Keeping pace withcriminals: Designing patrol allocation against adaptiveopportunistic criminals. In AAMAS 2015.