Evaluating a plan-recognition framework using tree automata · Plan recognition is the sub- eld of arti cial intelligence where we try to rec-ognize the plans of an autonomous agent

Evaluating aplan-recognition framework

using tree automata

Eric Jonsson

February 1, 2012Bachelor’s Thesis in Computing Science, 15 credits

Supervisor at CS-UmU: Johanna BjorklundExaminer: Pedher Johansson

Umea UniversityDepartment of Computing Science

SE-901 87 UMEASWEDEN

Abstract

Plan recognition is the task of inferring the possible plans an independentagent might have in mind from observing and analyzing the agents actions.Plan recognition have, among other things, practical applications within thefields of signal analysis and artificial intelligence.

In 2010 Johanna Bjorklund and Lisa Kaati developed a framework for planrecognition using weighted, unranked tree automata. This paper explains howtheir algorithm was implemented and evaluated in the context of StarCraft II,a real-time strategy computer game.

ii

Contents

1 Introduction 1

1.1 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Problem description . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Foundations in formal language theory 5

2.1 Context-free grammars . . . . . . . . . . . . . . . . . . . . . . . 5

2.2 Parsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.3 Chomsky normal form . . . . . . . . . . . . . . . . . . . . . . . 9

2.4 CYK extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.5 Plans, grammars and trees . . . . . . . . . . . . . . . . . . . . . 11

3 Implementation details 13

3.1 Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.2 The tree editor . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.3 The grammar compiler . . . . . . . . . . . . . . . . . . . . . . . 14

4 Evaluation 17

4.1 On StarCraft II . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.1.1 A game of economy . . . . . . . . . . . . . . . . . . . . 17

4.1.2 The technology tree . . . . . . . . . . . . . . . . . . . . 18

4.1.3 Hidden information . . . . . . . . . . . . . . . . . . . . 19

4.2 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.3.1 Spatial limitations . . . . . . . . . . . . . . . . . . . . . 23

4.3.2 Temporal limitations . . . . . . . . . . . . . . . . . . . . 25

4.4 Conclusion and future work . . . . . . . . . . . . . . . . . . . . 26

References 29

iii

iv CONTENTS

Chapter 1

Introduction

Plan recognition is the sub-field of artificial intelligence where we try to rec-ognize the plans of an autonomous agent by observing the actions it takes[7].This is useful is a number of contexts, from AI opponents in computer gamesto military surveillance and computerized language understanding[1].

To get an idea of plan recognition in action, consider Example 1.0.1.

Example 1.0.1: A man walks into a supermarket on Friday night and picksup chocolate-chip cookies and milk. A computer system observes the man andhis purchases, and deduces that the man is probably planning a quiet indoorevening rather than a night out at the pub.

Plans are seldom so clear-cut: from a set of actions, there are often multiplepossibilities of plans, as the next example shows. Imagine an observer listeningin to the conversation in Example 1.0.2.

Example 1.0.2: “Excuse me, could you tell me where the train ticket officeis?”

There are multiple reasons why someone might want to know where theticket office is: he might want to purchase a train ticket, query for departureinformation, or even ask the cashier out for a date! To deal with this issue,observers need more information. Us humans might ask a follow-up question,such as in Example 1.0.3.

Example 1.0.3: “Do you need a ticket towards Windsor or Quebec?”

“Windsor.”

When dealing with plans and actions, it is also often useful to considersub-plans, that is, plans that are “stepping stones” onto greater plans. In thecontext of the train-example above, we might consider the plan to buy a ticketto be a part of the greater plan of travelling to Windsor.

1

2 Chapter 1. Introduction

Furthermore, we differentiate between cases where the agent does its bestto make his intentions clear, (intended recognition) and the cases where itdoes not. (“keyhole” recognition) In the first case, a recognizer can makeassumptions, in the second it must instead deal with the fact that it mayhave incomplete information[6]. Of course, in some contexts, (such as warfare)agents can even do an effort to make its plans as difficult as possible to detect ortrigger false plans. These types of agents makes an already hard problem evenharder to the point where we simply choose to disregard these cases and insteadconcern ourselves with agents that are merely ignorant and/or unhelpful whenwe talk of keyhole recognition[3].

An extensive amount of work has been done in this area already, in par-ticular by Kautz, Allen and Perrault[7][1] in laying the foundations of planrecognition outside the scope of specific domains. Carberry[3] wrote an excel-lent survey of the topic in 2001, well worth the read.

1.1 Problems

Plan recognition, despite having received a fair amount of attention in the past,still struggles with several fundamental problems. In order to fully appreciatethe finer details of the different approaches taken in literature, let us list someof the challenges that plan recognition entails.

1. First, and perhaps foremost, we deal with the fact that plans can becomposed of multiple observations – just consider the bomb-making ter-rorist who constructs his bomb using readily-available over-the-counteringredients. Individual purchases of these items might by themselves bemeaningless, but when put in the context of each other they lead us tobelieve that there is a serious threat. As a general rule, a plan-recognitionsystem must therefore be able to analyse and recombine a large deal (ifnot all) of its input in order to make a good prediction.

2. As a consequence of Problem 1, plan-recognition systems face challengeswith regard to computational complexity versus descriptive power[2]. Asystem that performs complex but accurate predictions might be not befeasible to use on large datasets, but on the other hand systems thatperform simple predictions might lack the descriptive power to identifyimportant types of plans.

3. For a plan-recognition system to be of any use, it must be able to dealwith noise, that is, observations that is not necessarily part of any plan.

4. Some plans can be achieved in more than one way, and not every agent isrational or effective. For better descriptive power, a plan-recognition sys-tem should be able to identify plans even when the path there is differentthan the best, or obvious one.

1.2. Problem description 3

1.2 Problem description

Having introduced the field of plan recognition, we now consider the attemptby Bjorklund and Kaati[2] to deal with Problem 2 from Section 1.1: the factthat plan-recognition systems generally are computationally complex, i.e., theyeither use too much memory, take too much time to execute, or both. Theirapproach is essentially hierarchical divide-and-conquer using trees. (That is,connected acyclic graphs) Similar strategies has historically done very well inspeeding up other algorithms, their approach is explained in Chapter 2.

One issue with Bjorklund and Kaati’s algorithm stands out, however: whileit is sound in theory, there have been no known attempts at implementingit. An implemented prototype would allow further studies by automating testruns of the algorithm against large sets of observations that would otherwisebe difficult to study, and possibly identify corner cases where the algorithm isweak.

We therefore stake out these two goals of the study:

1. Write a software framework to help with the evaluation of the algorithmproposed by Bjorklund and Kaati. This framework should consist of thefollowing components:

(a) A plan editor that lets a user create and modify plans, to create, useand maintain a plan library.

(b) A grammar compiler that as its input takes a tree of plans and asoutput generate a grammar whose language describes them. (Again,to understand how this works, see Chapter 2.)

(c) A test application that lets a user input data, apply the algorithmand observe the results.

2. Evaluate the algorithm against a suitable corpus of test data in order toglean some deeper insight of its advantages and disadvantages, possiblyproposing modifications or extensions to it.

4 Chapter 1. Introduction

Chapter 2

Foundations in formallanguage theory

Before diving into plan recognition, it is important to have a solid foundation ofpreliminary knowledge. This chapter will outline and explain some importantconcepts, in particular on the topic of context-free grammars, how they areused, their relation to tree automata, how they can be operated upon and howalgorithms can be designed to work with them.

2.1 Context-free grammars

Context-free grammars have ever since their introduction by Noam Chomskyin the 1950’s been an important linguistic tool for describing the structure ofsentences in both natural and artificial languages. Formally, the context-freegrammar G is defined by the 4-tuple G = (V,Σ, R, S)[10], where:

1. V is a finite set of non-terminals or variables.

2. Σ is a finite set of terminals, commonly referred to as the alphabet of thegrammar.

3. R is a finite relation from V to (V ∪ Σ)∗ called the rewrite rules orproductions of the grammar.

4. S, a member of V , is the start symbol.

Informally, a rule is often written like V → w, where V is a non-terminaland w is a (possibly empty) string of terminals and non-terminals. The emptystring is denoted by the Greek letter epsilon, ε. Furthermore, when listingterminals, non-terminals and rules, terminals are often written in lower-caseand non-terminals in uppercase so as to make it easier to tell them apart.

5

6 Chapter 2. Foundations in formal language theory

The start symbol, S, is often marked with an arrow on the left-hand side todistinguish it from other variables.

A common example is the following rule set:

Example 2.1.1:

⇒S→ X

X→ aXb

X→ ε

To generate a string using this grammar, non-terminals are repeatedly re-placed with terminals and non-terminals from the right-hand side of any match-ing rules, beginning with the start variable. Eventually, when no non-terminalsremains, (that is, the string only consists of terminals) the resulting string issaid to be included in the language defined by the grammar.

To demonstrate how the above grammar generates the string ab, performthe following steps:

1. Replace S with X. (Result: X)

2. Replace X with aXb. (Result: aXb)

3. Replace X with ε. (Result: ab)

It is easy to see that by performing step two an arbitrary number of times,this grammar can not only generate ab, but also aabb, aaabbb, or in generalanbn where n ≥ 0.

Another way to depict the same sequence of actions is with a parse tree[10].In a parse tree, the root is the starting symbol, and its children are the non-terminal symbols it evaluates into. At the bottom of the tree we find theleaves being the terminals. These terminals make up the ordered sequence ofcharacters in the parsed string, called the yield. Figure 2.1 shows the parsetree for the grammar in Example 2.1.1 for the string aaabbb.

Figure 2.1: The parse tree for the grammar in Example 2.1.1 when applied tothe string aaabbb.

2.2. Parsing 7

2.2 Parsing

Once we’ve understood how context-free grammars can be used to generatestrings, the next step is to solve the inverse problem – answering “yes” or “no”to the question “can a given grammar generate a given string”? In fact, wecan. This is called parsing, and there are numerous algorithms that does thiswith varying space- and time-requirements.

For the purposes of plan-recognition, we will use a modification of theCocke-Younger-Kasami (CYK) algorithm. To demonstrate how the unmod-ified algorithm works, let us see if the string “Jeff trains geometry students”[4]can be generated from the following grammar:

⇒S→ NVp

V→ NN

N→ NN

Vp → VN

N→ students|Jeff|geometry|trainsV→ trains

The CYK-algorithm employs bottom-up parsing, which means that eachtoken in the string is checked to see if it can be accepted into structures ofa higher order. To store intermediate results, it uses the lower half of a two-dimensional table, (see Table 2.1) where the cell at row i, column j holds thevariables that can generate each substring of length i starting at index j.

4321

Jeff trains geometry students

Table 2.1: The empty CYK table for the phrase “Jeff trains geometry students”.

We fill the table, starting with row 1, by finding out which of the productionrules that match the tokens (words) in the input string.

For the second row, we consider the substrings of length two. Note thatboth V and N can generate NN, and that Vp also generates VN.

The third row is a little trickier – we need to consider all possible orderedpair of tokens for each substring of length three: this means considering VV,VN, VVp, NV, NN and NVp for the first cell alone, as well as VV, NV, VpV, VN, NNand VpN for the second. As before, we fill in the variables that matches any ofthese.


4321 N N, V N N


Table 2.2: CYK table, first row filled in.

432 V, N V, N, Vp V, N

1 N N, V N N


Table 2.3: CYK table, second row filled in with substrings of length two.

43 Vp, V, N, S Vp, V, N2 V, N V, N, Vp V, N

1 N N, V N N


Table 2.4: CYK table, third row filled in with rules matching all possibleordered substrings of length three.

Row four (and those above, in case of longer strings) is filled in the sameway as row three. The algorithm ends when the single last cell in the topmostrow has been filled in. The question “can this grammar generate this string?”is then answered with “yes” if and only if the topmost cell contains the startsymbol, otherwise with “no”.

4 S, Vp, V, N3 Vp, V, N, S Vp, V, N2 V, N V, N, Vp V, N

1 N N, V N N


Table 2.5: CYK table fully filled, start symbol S present in the top row.

2.3. Chomsky normal form 9

2.3 Chomsky normal form

In Chapter 2.2, we observed how the CYK-algorithm combined variables fromadjacent cells into pairs, and then attempted to match those pairs with theones in the set of rules for the grammar. But what if the grammar had rulesthat comprises of more than two variables? Those rules would obviously nevermatch any of the pairs the CYK-algorithm tried, and so the algorithm wouldfail.

To amend this, the CYK-algorithm demands that the context-free grammaris on Chomsky normal form, (CNF) which means that every rule are of the form:

A→ BC or

A→ a or

S→ ε

Furthermore, neither B nor C may be the start symbol. Fortunately, thereare several algorithms that converts a given grammar to Chomsky normal formin such a way that they recognize the same language[10]. We introduce onehere:

1. The start symbol S can occur on the right-hand side in one or more rules,which is forbidden by CNF. To amend this, we introduce a new variableS0, make it the new start symbol, and add the rule S0 → S to the ruleset.

2. Eliminate all ε rules that does not involve the start symbol, that is, allrules of the form A → ε where A 6= S. Then, for each rule with A on theright-hand side, add a new set of rules containing the possible combina-tions of A replaced or not replaced with the empty string. Furthermore,if a rule has A as a singleton on its right-hand side, add a new rule A→ εunless this new rule has already been removed earlier during the algo-rithm.

For example, consider the following rule set:

Example 2.3.1:

⇒S→ AbA

A→ ε

We identify A→ ε as an epsilon rule to be removed. We remove it, thenlook for all occurrences of A in all other rules and find them in the ruleS→ AbA. We replace this rule with the following rule set:


S→ AbA

S→ Ab

S→ bA

S→ b

3. Eliminate all unit rules A→ B such that A, B ∈ V. A “unit rule” is heredefined as a rule with a single variable on the right-hand side and noterminals. To remove a rule A→ B, for each rule B→ U add a rule A→ U

unless, again, this rule has already been removed during the process.

4. Next, we replace all rules A → u1u2 . . . uk where k ≥ 3 and each ui is avariable or terminal symbol with the “chain” of rule rules A→ u1A1, A1 →u2A2, . . . , and Ak−2 → uk−1uk. The Ai’s are all new variables. This stepguarantees that every rule have no more than two variables or terminalson the right-hand side.

2.4 CYK extensions

The CYK-algorithm, as described as in Section 2.2, is used every time a newobservation is recorded. Performance-wise, this would be a huge hit to analready-slow algorithm were it not for the fact that every new observation isappended to the stream of input. This allows for the following optimization,where parts of the previously used CYK table is re-used for the next iteration.

Let w be the observation window size, an integer that represents how manytokens backwards in the input stream we are to consider for relevance in somepossible plan. If omitted, we would be using every recorded piece of observationin each iteration, which grows impractical as the size of the input grows large.In practise, this window size needs to be set to some pre-defined value.

When a new observation is about to be added to the input stream, wefirst consider the length of the resulting string. If it would be longer than theobservation window size w, we remove the oldest observation along with itscorresponding column in the CYK table from consideration prior to addingthe new one, see Table 2.6. This ensures that the length of the string we areanalyzing never exceeds w.

4 S, Vp, V, N3 Vp, V, N, S Vp, V, N2 V, N V, N, Vp V, N

1 N N, V N N


Table 2.6: The CYK table from Section 2.2 with first column dropped.

2.5. Plans, grammars and trees 11

With the first column dropped, there’s now room in the table for the newobservation. We create up to w new, empty cells in the CYK table along thediagonal, as is shown in Table 2.7.

43 Vp, V, N2 V, N, Vp V, N

1 N, V N N

trains geometry students today

Table 2.7: CYK table with old data discarded and w new empty cells addedalong the diagonal.

With the CYK table prepared, the next task is now parsing the string “traingeometry students today”. This string has many substrings in common withthe previous string “Jeff trains geometry students”, which is why we are ableto save the intermediate results. When using this technique, we only need toperform w intermediate parsing steps: filling in the recently added w cells alongthe diagonal, (see Table 2.7) as described in Section 2.2.

2.5 Plans, grammars and trees

Having introduced the subject of plan recognition in Chapter 1 and revisitedthe foundations of formal language theory in Sections 2.1, 2.2, 2.3 and 2.4, weare ready to explain the approach taken towards plan-recognition by Bjorklundand Kaati[2].

Under this approach, plans consists of observations connected with the log-ical connectives conjunction and disjunction in a treewise manner. Considerthe simple example in Figure 2.2.

Figure 2.2: A simple plan: three observations (A, B, and C) connected by aconjunction (D) to a fourth observation (E) with a disjunction. (F)


– In Figure 2.2, A, B, C and E are all observations. Every observation is aleaf in the plan tree.

– D and F are plans. In the case of D, a sub-plan since it is a child of F inthe plan tree.

– Every plan is associated with a logical connective, AND (∧) in the case ofD, OR (∨) in the case of F. For plan D, this means that A, B and C allhave to be observed for plan D to be considered fulfilled. For plan F, itis sufficient with either D or E being observed.

Recall that we in Section 2.1 briefly mentioned the yield of a tree. Here, letit be defined as the string of symbols that labels its leaves when read from leftto right[2], or more formally a function whose domain are trees and its imagethe ordered set of leaves from its input. Using the tree t in Figure 2.2 as anexample:

yield(t) = {A,B,C,E}

The yield function is important, because it ties string languages (expressableby context-free grammars) and regular tree languages (expressable by finite treeautomata) together in a fundamental way:

1. Every string language that can be generated by a context-free grammaris the yield of a tree language accepted by some finite tree automaton[2].

2. Every regular tree language that can be recognized by a finite tree au-tomaton is the parse tree of a string language generated by some context-free grammar[2].

This means that if we wish to recognize the yield of a plan tree, we can con-vert our tree automata to context-free grammars and apply the CYK-algorithmon them as explained in Sections 2.2 and 2.4.

Chapter 3

Implementation details

This chapter describes in some detail how the proof-of-concept software wasimplemented, what the major components are, what technology they use, andhow the software can be improved and expanded upon.

3.1 Technology

The proof-of-concept software was written in the Java programming language.This choice was made mainly due to the following reasons:

– Unlike most other languages, everyone involved in the project were com-fortable reading and writing Java code.

– It is cross-platform, and so ran without problem and without modifica-tions on the different machines used in development. (Ubuntu Linux andMac OS X)

– There were high-quality third-party libraries available for Java that short-ened development time significantly.

Java includes a graphical user interface (GUI) toolkit called Swing thatwas used to build the tree editor. (See Section 3.2) In addition, to visualizethe plan trees, the third-party library JUNG (Java Universal Network/GraphFramework) was used.

3.2 The tree editor

A major component of the software is the tree editor. (Sometimes called theplan editor) It is a graphical tool that allows an operator to create, store, loadand modify plans.

13

14 Chapter 3. Implementation details

Recall that in Section 2.5, we explained how a plan tree consists of obser-vations connected with conjunctions and disjunctions in a tree structure withthe plan as the root node. Every leaf (observation) has a label that identifiesit, and tree nodes have properties such as name, logical operator and so forth.The tree editor contains tools to modify these properties, as well as tools tochange the structure of the tree – adding and removing vertices, connectingand disconnecting vertices and so on. Furthermore, plans created can be savedinto (and restored from) XML files, forming a plan library an analyst can uselater. Figure 3.1 shows the main workbench area of the software.

Figure 3.1: The tree editor workbench, where an analyst can create, save, loadand modify plan trees.

3.3 The grammar compiler

The next major component of the software is the grammar compiler. At itsheart, it transforms (compiles) an internal representation of a plan tree into acontext-free grammar according to the following algorithm:

1: Push R, the root of the plan tree, onto a stack S2: while S is not empty do3: Let P ← the top plan popped from S4: Let C1 . . . CN ← the child nodes from P5: if P is an AND-node and N > 0 then6: Add the rule P→ C1 . . . Cn to the rule set of the grammar.7: else if P is an OR-node then8: for i = 1→ N do9: Add the rule P→ Ci to the rule set of the grammar.

10: end for11: end if12: Push C1 . . . Cn onto S13: end while

3.3. The grammar compiler 15

Once the rule set has been created, the grammar is wrapped up:

1. Its start state is set to the tree root.

2. The terminals are set to be the leaves of the tree, and the non-leavesbecomes the non-terminals.

At this stage, the grammar compiler also performs the task of convertingthe resulting grammar to Chomsky normal form as described in Section 2.3.After this step, the grammar is ready to be used.

16 Chapter 3. Implementation details

Chapter 4

Evaluation

With the prototype in hand, we now turn our eyes towards evaluating thealgorithm it implements. The major task here is finding a corpus that is bigenough to be meaningful, but still simple enough to make sense of any resultsthat emerge. As will be shown, it turns out that the real-time strategy gameStarCraft II can act as our evaluation platform satisfactory.

4.1 On StarCraft II

Regarding StarCraft II, there are mainly three factors that combine to make ita good evaluation platform. These factors are discussed in Sections 4.1.1, 4.1.2and 4.1.3.

4.1.1 A game of economy

At its core, StarCraft II is a game of economy. While the end goal of the gameis to defeat the opponent in fictional combat, a player needs units to do so.These units typically represents soldiers, and much like the pieces of a chessgame they have their advantages and disadvantages towards each other. Themain difference is that a player does not start the game with very many ofthese units – they are created in production facilities over the course of thegame.

So in order to defeat the opponent, it is in a players interest to have asmany (and as powerful) of these units as possible so as to outnumber the unitsof the opponent. However, these units do not come without a cost – the gameidentifies three types of resources, items that a player can gather over the courseof the game, and each unit produced costs the player a specified amount of theseresources. By extension, it is then in the best interest of the players to gatheras much of these resources as possible, in order to produce a bigger and morepowerful army than the opponent.

17

18 Chapter 4. Evaluation

Next a circular dependency is introduced, see Figure 4.1: resources canonly be gathered by certain worker units, relatively weak ones in a militarysense. Each player starts the game with a fixed number of these as well as asmall amount of resources, and the only way a player can increase the resourceincome rate is to produce more of these (weak) units and put them to work.

Figure 4.1: A circular dependency: worker units are needed to gather resources,and resources are spent to produce worker units.

This demonstrates a core concept of StarCraft II: that of income management[8].A player can at any point in time choose one of two options:

1. Create worker units in order to bolster resource income rate,

2. or create regular, military units that can achieve the end goal of defeatingan opponent in battle.

In other words, production of worker units can be thought of as an invest-ment – a decision that leaves a player military weak at this instant in time, butpossibly more powerful as the game goes on and the worker starts to recouphis own cost. By the same token, the production of military units does notaffect the resource income rate, but obviously leaves the player with a strongermilitary force. Players need to weigh the advantages and disadvantages of bothchoices against each other with regard to what the other player is doing[8] –a player that emphasises military units over worker units must capitalize onthese lest the opponents superior income allows him to out-produce him laterin the game, and a player that sacrifices military strength for future incomemight find himself losing the game to military might earlier than expected.

4.1.2 The technology tree

In StarCraft II, not all the production facilities or units discussed in Section4.1.1 are immediately available: some require the players to already controlother facilities, constructing what players call the technology tree. A (partial)technology tree can be seen in Figure 4.2.

The existence of the technology tree is an important one, because it providesan opportunity for opponents to gain insight into a players plan. A playeris discouraged to construct all components in the tree as soon as possible,because such an action would consume resources that could otherwise be putto better use. Using Figure 4.2 as an example, it tells us that the observationof a Tech Lab immediately tells the observer that Barracks exists somewhere(since it is a prerequisite for the Tech Lab) and could signify the production of

4.1. On StarCraft II 19

Figure 4.2: A partial technology tree from StarCraft II. Rectangles representsproduction facilities, ellipses represents units.

Marauders. Likewise, if a player observe a Marauder, it reveals the existenceof both Barracks and Tech Lab but says nothing about, for example, theexistence of Marine or Starport. On the other hand, a player does not likelywant to forgo construction of more expensive facilities completely, since thesefacilities rewards the player with access to more powerful units. Just as witheconomy, (discussed in Section 4.1.1) players must weigh the advantages anddisadvantages of both options and weigh into their decision what the opponentis doing.

4.1.3 Hidden information

The one important property that finally binds both economy (discussed inSection 4.1.1) and technology tree (Section 4.1.2) together is the fact thatStarCraft II is a game of imperfect information, i.e. players are not immediatelyaware of the full game state[9]. The concept of perfect or imperfect informationis an essential one in areas such as game theory and economics.

This property is achieved by not letting the players have full vision of the en-tire game area, but only in the immediate area around their units and buildings.Since information such as the number of worker units and which productionfacilities a player has gives critical information about their plans, it is in thebest interest of players to deny this information to their opponents.

It is easy to see how the concept of perfect and imperfect information cor-responds to the difference between intended and keyhole recognition from Sec-tion 1. If the game state is fully know, as in (for example) the game of chess,finding the winning strategy simply becomes a search problem, and (for Star-Craft II) there is even a good chance that simple, greedy algorithms would dovery well since there is typically a window of time between the initiation of aplan and when it comes to fruition.

On the other hand, if the game state is not fully known, our agents aredealing with imperfect information and we are now given opportunities to draw


conclusions from our observations. Examples include:

– If unit A precedes unit B in the technology tree (see Section 4.1.2) andwe observe unit B, we can conclude that unit A exists.

– The concept of noise suddenly becomes meaningful. It is fair to assumethat every action an opponent takes is intended to help him win the game,and so every action is part of some plan. With imperfect information weare unable to see every action and thus fit every action into a plan, theseunaccounted actions becomes noise among the actions we can map to aplan.

4.2 Method

Using StarCraft II as a testing environment has the upside that a lot of (ad-mittedly, non-rigorous) strategy analysis has been done and published onlineby the player community. This allowed us to:

1. Quickly compile a set of the most popular plans and strategies used byhighly-skilled players.

2. Download videos (known as replays) of these players playing the gameusing these strategies and, from one chosen player’s point of view, encodeit into a sequence of observations along with a time stamp.

Since the game features a large amount of parameters which alters thepossible applicable plans, we constrained ourselves to very specific criteria forgame replays:

1. Players are ranked by a points system, where points are awarded or de-ducted from a player when she wins or loses a game. We only concernourselves with games from very good players, defined as the top 2% ofthe player population in terms of points. This helps eliminate bad plans,since good players can be expected to use close to optimal plans.

2. StarCraft II features three playable races, giving a total of nine possiblerace combinations for one versus one games – we arbitrarily choose oneof these combinations, the human Terrans versus the alien Zerg for ouranalysis.

3. From these two Terran and Zerg players, we choose the Zerg players pointof view: we concern ourselves with trying to find out what the Terranplayer’s plans are, rather than the other way around.

4. StarCraft II can be played in a variety of game environments called maps.We arbitrarily choose the map Xel’Naga Caverns for our analysis.

4.2. Method 21

Having laid out the parameters for the replays, a small corpus matchingour parameters were downloaded from a major community website[5]. Un-fortunately, at the time there were no known tools available to automaticallyconvert these replays into timestamped lists of observations, so this had to bedone manually, severely limiting the size of the corpus. Once the lists of obser-vations had been acquired and plan trees created, the software was run withthis resulting dataset as input and generated warnings for analysis. Example4.2.1 illustrates how events in the game maps onto observations and warningsin the software.

Example 4.2.1: Figure 4.3a shows an in-game situation where two Zerg units(circled in the top right) spots a Terran SCV unit. In Figure 4.3b, we can seethe observation is mirrored in the observation panel of the analysis software.(Circled) The software is configured to recognize a Barracks, a Factory and aTech Lab as signs of a Hellion threat coming soon, meaning the observationof SCV do not trigger any warnings.

(a) (b)

Figure 4.3: Two Zerg units spotting a Terran SCV unit, with the observationmirrored in the analysis software.


Figure 4.4a shows a screenshot from the same scenario just a couple ofminutes later, where the Zerg player is sending a scouting unit (circled) up aramp to spot what his opponent is doing. As no other observations has beenmade since the first SCV a couple of minutes ago, the state of the analysissoftware (Figure 4.4b) has not changed.

(a) (b)

Figure 4.4: A scouting Zerg Drone unit. With no further observations at thispoint, the state of the analysis software is unchanged.

In Figure 4.5a, the Zerg scout has observed the final piece of the puzzle,a Barracks, Factory and Tech Lab. The series of observations now parsescorrectly as a possible Hellion threat, so the user is alerted with a big, boldwarning in the notification area of the software.

(a) (b)

Figure 4.5

4.3. Results 23

4.3 Results

This section documents the results from the test runs laid out in Section 4.2. Wediscuss situations in which the implemented algorithm fails, provide illustrativeexamples, and suggest workarounds or fixes where applicable.

4.3.1 Spatial limitations

Recall that in our plan recognition algorithm, we model plans as nodes in atree and that our plan tree (or rather our grammar built from said tree) is re-used over and over again to evaluate observations as they are streamed in. Inparticular, this means an observation is treated the same regardless of where inour two-dimensional game-world they take place, provided they have the sameidentifying name.

This is obviously problematic, since where an observation takes place mat-ters. For example, a man firing a rifle on a shooting range likely have verydifferent plans than a man firing a rifle in a schoolyard or urban area. In ourStarCraft-context, production facilities (see Section 4.1.1) located in the mid-dle of our opponents base, (while noteworthy in itself) likely means somethingdifferent than if those facilities were discovered in a remote, hidden locationnot meant for us to find.

Workaround

Of course, one could argue that it is not the parsing algorithm itself that is atfault here, merely that it is not fed enough data. A simple workaround to thisproblem is to come up with a way to encode spatial data in the observationname, so that for example the observation Command Center located at ouropponents main base becomes Command Center mainbase as to differentiateit from Command Center natural, meaning a command center placed at ouropponents natural expansion place. While this is likely a sufficient way todeal with situations arising in our very limited game-world, it is neither avery efficient, elegant or generic enough solution for other, more demandingenvironments.

A better solution

A better solution to the spatial problem is to take a page from the geographicalinformation systems-book (GIS) and devise extra, proper metadata for each ofour observations, a shape. First of all, for our purposes we define a shape as:

1. A set of coordinates H = (x1, y1), (x2, y2), . . . , (xn, yn).

2. A rule to classify any given coordinate C = (x, y) as (at a minimum)either inside or outside the shape using the coordinates in H.


Figure 4.6: Two simple shapes, a circle and a rectangle. Both are definedusing two coordinates each, (marked with crosses) but with different rules tospecify where the shape border (black) is drawn. The grey area indicates points“inside” the shape.

Figure 4.6 illustrate two examples of simple shapes, the circle and the rect-angle. We can easily come up with other useful shapes – for example, for manyapplications it is likely that a single coordinate, a point, is sufficient while otherapplications might demand more complicated shapes of observations. This sys-tem is generic enough to be easily extended to whatever shapes the applicationdemands.

Secondly, we extend our plan tree nodes (in addition to the observationname already present) to also incorporate:

1. A shape. This literally means defining the area where observations of thisclass can take place.

2. A spatial rule, such as “inside”, “outside”, “touches”, “intersects”, etc.

Third, we simply modify the parsing algorithm to additionally check forspatial matches. Example 4.3.1 illustrates the idea.

Example 4.3.1: Figure 4.7 depicts one of the two “base” areas of the mapused in testing, (see Section 4.2) with two shapes and their set of defining coor-dinates highlighted in grey, as well as the production facility Command Center

represented by a dark circle.

In StarCraft II, a player starts the game with a Command Center facility,which means that the observation of this building in this spot says very littleof our opponents plan – she has not placed the building there by choice.

Figure 4.8 however, is much different – we here see an additional CommandCenter (again, represented by a dark circle) placed in the square shape ofthe natural expansion area, giving away much more information about ouropponents plan.

As Example 4.3.1 shows, by introducing two different shapes covering twodifferent areas on the map, we are able to differentiate between identical build-ings placed in these two locations, giving us a much more powerful plan recog-nition system for these kind of applications where spatial location matters.

4.3. Results 25

Figure 4.7: A sample from the map used in testing, with two shapes represent-ing the main base and natural expansion areas, and a circle representing thestarting location of the Command Center facility.

Figure 4.8: The same figure as Figure 4.7, but with an additional CommandCenter placed within the square shape of the natural expansion area.

4.3.2 Temporal limitations

Another issue that arose during testing was temporal limitations, the fact thatthe plans of an agent can depend on when its actions are performed. Theplan-recognizing algorithm does not support this: even though identical ob-servations, and even identical series of observations can trigger different plansdepending on the context of other observations, there is no notation of any“time windows” when plans are active.

Recall that in Section 4.1.1 we explained how the game uses three types ofresources. As testing showed, it turns out that a player is (at some point inthe game) going to harvest resources from all resource fields she has available– the difference is merely when the player starts doing so. Yet it is important,since it will directly affect what units she will be able to produce, and thuswhat type of threat she poses to the other player.

In a real-life scenario, we can imagine that a man that enters the frontdoor of a house on a weekday afternoon is merely a man that lives there, but


a man that sneaks in at 2 AM during a holiday is a burglar. Thus, in someapplications, timing matters.

Proposed solution

A simple solution to the temporal problem is to add metadata to our obser-vations and plans, just like we did in Section 4.3.1, and modify the parsingalgorithm to take them into account:

1. Let each plan carry two timestamps, Tstart and Tend such that Tstart ≤Tend holds.

2. Let each observation also carry a timestamp Tobs that represent the pointin time where the observation took place.

3. Modify the parsing algorithm so that (in additional to other match-rules)a plan only matches an observation if Tstart ≤ Tobs ≤ Tend.

This lets us create “window of opportunities” where plans are only activefor a certain amount of time, giving us finer-grained control of our plan libraryand (for applications such as these) hopefully much better plan-recognitionaccuracy.

4.4 Conclusion and future work

From the paper by Bjorklund and Kaati[2], it is clear that the basic algorithmis highly efficient in streaming situations and would be a very good candidateto evaluate player’s plans in real time strategy games such as StarCraft II. Evenso, the algorithm was found lacking to the point of it being unusable, mainlysince it cannot efficiently1 capture the different timing windows for plans thatplayers make such great use of or differentiate between observations that takesplace in different spatial locations.

Suggestions on how to fix these problems were presented in Section 4.3, andshould form the base task for a follow-up study on the same topic. We find itlikely that once the huge show-stopper problems of spatial and temporal limi-tations are overcome, more interesting results would emerge as the algorithmbecomes better tuned to this specific problem.

Even so, there are implementation-specific optimizations to be made thatcan greatly improve performance. Recall that:

1. The parsing step is performed up to w times for each new observation,where w is the observation window size.

1Creating a very large set of identical plans that only differs on their timing window ispossible, but is so inefficient (bordering on the absurd) that it was never considered a viablesolution in Section 4.3.2.

4.4. Conclusion and future work 27

2. When the grammar representing the plan tree is converted to CNF, eachdisjunction creates N new rules where N is the number of children forthat node in the plan tree. The size of the grammar can therefore growexponentially.

3. Each parsing step searches the entire rule set of the grammar for matchingrules.

This means that searching the list of grammar rules becomes a major perfor-mance bottleneck. Clearly this is a place where even very small improvementshave the potential to make a big difference in overall speed, such as exchangingthe rule list for a rule hash table.

Figure 4.9: A hash table containing 13 rules, sorted into 4 buckets.

Figure 4.9 shows a collection of 13 rules, split over 4 “buckets”, or lists. Byusing a rudimentary hash function2, (namely putting all rules with a matchingleft-hand side in the same bucket) we are able to narrow down the search andfind matching rules much quicker.

2There are better hash functions available, but this one illustrates their advantages in asimple way.


References

[1] James F. Allen and C. Raymond Perrault. Analyzing intention in utter-ances. Artificial intelligence, 15(3):143–178, 1980.

[2] Johanna Bjorklund and Lisa Kaati. Weighted unranked tree automataas a framework for plan recognition. In Proceedings of 13th InternationalConference on Information Fusion, 2010.

[3] Sandra Carberry. Techniques for plan recognition. User Modeling andUser-Adapted Interaction, 11(1):31–48, 2001.

[4] Sariel Har-Peled and Madhusudan Parthasarathy. Cyk parsing algorithm.Lecture on the theory of computation, found online at http://www.cs.

uiuc.edu/class/sp09/cs373/lectures/lect_15.pdf, 2009.

[5] Curse Inc. Sc2replayed website. http://www.sc2replayed.com, 2010.

[6] Henry A. Kautz. A formal theory of plan recognition and its implementa-tion. Reasoning about plans, pages 69–126, 1991.

[7] Henry A. Kautz and James F. Allen. Generalized plan recognition. In Pro-ceedings of the Fifth National Conference on Artificial Intelligence, 1986.

[8] Sean Plott. Day[9] daily #194. Accessed at http://blip.tv/day9tv/

day-9-daily-194-newbie-tuesday-drone-timing-4228999, October2010.

[9] Stuart J. Russell and Peter Norvig. Artificial Intelligence: A ModernApproach, 2nd Ed. Prentice Hall, Englewood Cliffs, NJ, 2002.

[10] Michael Sipser. Introduction to the Theory of Computation. Course Tech-nology, second edition, 2005.

29

Documents

Evaluating a plan-recognition framework using tree automata · Plan recognition is the sub- eld of arti cial intelligence where we try to rec-ognize the plans of an autonomous agent