GAMES INEXTENSIVE AND STRATEGIC FORMS · Chapter2 GAMES INEXTENSIVE AND STRATEGIC FORMS SERGIU HART' TheHebrew University ofJerusalem Contents O. Introduction 1. Games inextensive

Chapter 2

GAMES IN EXTENSIVE AND STRATEGIC FORMS

SERGIU HART'

The Hebrew University of Jerusalem

Contents

O. Introduction1. Games in extensive form2. Pure strategies3. Games in strategic form4. Mixed strategies5. Equilibrium points6. Games of perfect information7. Behavior strategies and perfect recallReferences

202025262829293240

'Based on notes written by Ruth J. Williams following lectures given by the author at StanfordUniversity in Spring 1979. The author thanks Robert J. Aumann and Salvatore Modica for someuseful suggestions..,

Handbook of Game Theory, Volume 1, Edited by R.J. Aumann and S. Hart@ Elsevier Science Publishers BY., 1992. All rights reserved

L

20 S. Hart

O. Introduction

This chapter serves as an introduction to some of the basic concepts that areused (mainly) in Part I ("Non-Cooperative") of this Handbook. It contains,first, formal definitions as well as a few illustrative examples, for the followingnotions: games in extensive form (Section 1), games in strategic form (Section3), pure and mixed strategies (Sections 2 and 4, respectively), and equilibriumpoints (Section 5). Second, two classes of games that are of interest arepresented: games of perfect information, which always possess equilibria inpure strategies (Section 6), and games with perfect recall, where mixedstrategies may be replaced by behavior strategies (Section 7).

There is no attempt to cover the topics comprehensively. On the contrary,the purpose of this chapter is only to introduce the above basic concepts andresults in as simple a form as possible. In particular, we deal throughout onlywith finite games. The reader is referred to the other chapters in this Handbookfor applications, extensions, variations, and so on.

1. Games in extensive form

In this section we present a first basic way of describing a game, called the"extensive form". As the name suggests, this is a most detailed description of agame. It tells exactly which player should move, when, what are the choices,the outcomes, the information of the players at every stage, and so on.

We need to recall first the basic concept of a "tree" and a few relatednotions. The reader is referred to any book on Graph Theory for furtherdetails.

A (finite, undirected) graph consists of a finite set V together with a set A ofunordered pairs of distinct members I of V. See Figure 1 for some examples ofgraphs. An element v E V is called a vertex or a node, and each {VI' V2} E A isan arc, a branch or an edge ("joining" or "connecting" the vertices VI and V2).Note that A may be anything from the empty set (a graph with no arcs) to theset of all possible pairs (a "complete" graph2). An (open) path connecting thenodes VI and Vrn is a sequence VI' V2'. . . , vrn of distinct vertices such that{VI' V2}, {V2' V3},...' {vrn-I' vrn} are all arcs of the graph (i.e., belong to A).A cycle (or "closed path") is obtained when one allows VI = vrn in the abovedefinition.

'Note that neither multiple arcs (between the same two nodes) nor "loops" (arcs connecting anode with itself) are allowed.

2The complete graph with n nodes has n(n - 1)/2 arcs.

Ch. 2: Games in Extensive and Strategic Forms 21

a

b

d ea. b. c. d.

w (~Figure 1. (a) A graph: V={a,b,c,d,e}; A={{a,b}, {a,c}, {c,d}, {c,e}}. (b) A graph:V= {a, b, c, d}; A = {{a, b}, {b, cD.

A tree is a graph where any two nodes are connected by exactly one path.See Figures 2 and 3 for some examples of trees and "non-trees", respectively.It is easy to see that a tree with n nodes has n - 1 arcs, that it is a connectedgraph, and that it has no cycles.

Let T be a tree, and let r be a given distinguished node of T, called the rootof the tree. One may then uniquely "direct" all arcs so they will point awayfrom the root. Indeed, given an "undirected" arc {v I' vJ, either the uniquepath from r to v2 goes through VI - in which case the arc becomes the orderedpair (VI' V2) - or the unique path from r to VI goes through V2- and then thearc is directed as (V2' VI)' The root has only "outgoing" branches. All nodeshaving only "incoming" arcs are called leaves or terminal nodes; we will denoteby L ==L(T) the set of leaves of the tree T. See Figure 4 for an example of a"rooted tree".

Figure 2. A tree.

22 S. Hart

a

a b

,n. /\,d

(a) (b)

Figure 3. (a) Not a tree (two paths from a to e). (b) Not a tree (no path from a to b).

a

c

d

Figure 4. A rooted tree: root = a; leaves = d, e, f.

We can now formally define an n-person game in extensive form, r, asconsisting of the following: 3

(i) A set N = {I, 2, . . . , n} of players.(ii) A rooted tree, T, called the game tree.

(iii) A ~artition of the set of non-terminal nodes4 of T into n + 1 subsetsdenoted p), p\ p2, . . . , pn. The members of po are called chance (or, nature)nodes; for each i EN, the members of pi are called the nodes of player i.

(iv) For each node in po, a probability distribution over its outgoingbranches.

(v) For each i E N, a partition of pi into k(i) information sets,Vii, V~,..., V~(iP such that, for eachj=1,2,... ,k(i):

'This definition is due to Kuhn (1953); it is more general than the earlier one of von Neumann(1928) [see Kuhn (1953, pp. 197-199) for a comparison between the two].

4A non-terminal node is sometimes called a "move".


(a) all nodes in U~ have the same number of outgoing branches, andthere is a given one-to-one correspondence between the sets ofoutgoing branches of different nodes in U~;

(b) every (directed) path in the tree from the root to a terminal nodecan cross each U~ at most once.

(vi) For each terminal node t E L(T), an n-dimensional vector g(t) =(gl(t), g2(t), . . . , gn(t)) of payoffs.

(vii) The complete description (i)-(vi) is common knowledge among theplayers.5

One can imagine this game r as being played in the following manner. Eachplayer has a number of agents, one for each of his information sets [thus i hask(i) agents]. The agents are isolated from one another, and the rules of thegame [i.e., (i)-(vii)] are common knowledge among them too. A pial of rstarts at the root of the tree T. Suppose by induction that the play hasprogressed to a non-terminal node, v. If v is a node of player i (i.e., v E pi),

then the agent corresponding to the information set U~ that contains v choosesone of the branches going out of v, knowing only that he is choosing anoutgoing branch at one of the nodes in U~ [recall (v) (a)]. If v is a chance node(i.e., v E po), then a branch out of v is chosen according to the probabilitydistribution specified for v [recall (iv); note that the choices at the variouschance nodes are independent]. In this manner a unique path is constructedfrom the root to some terminal node t, where the game ends with each player ireceiving a payoff gi(t).

Remark 1.1. The payoff vectors g(t) are obtained as follows: to each terminalnode tEL there corresponds a certain "outcome" of the game, call it a(t). Thepayoff gi(t) is then defined as ui(a(t)), where Ui is a von Neumann-Morgen-stern utility function of player i. As will be seen below, the role of thisassumption is to be able to evaluate a random outcome by its expected utility.

Example 1.2 ("Matching pennies"). See Figure 5: N = {I, 2}; root = a; po=

0; pi= U: = {a}; p2

= ui = {b, c}; payoff vectors (gl(t), g2(t)) are writtenbelow each terminal node t. Note that player 2, when he has to make hischoice, does not know the choice of player 1.7 Thus both players are in a

'That is, all players know it, each one knows that everyone else knows it, and so on; see thechapter on 'common knowledge' in a forthcoming volume of this Handbook for a formal treatmentof this notion.

60ne distinguishes between a game and a play: the former is a complete description of the rules(i.e., the whole tree); the latter is a specific instance of the game being played (i.e., just one pathin the tree).

7This shows the role of information sets; the game changes dramatically if player 2 knows atwhich node he is (b or c) when he has to make his choice - he can then always "win" (i.e., obtain apayoff of 1).

I

24

d

(1,-1) (-1,1 )

Figure 5. The game tree of Example 1.2.

h j(2,0,0) (0,2,0) (0,2,3)

k Lop q(1,1,1) (0,0,0) (1,2,3) (2,0,0) (0,1,-1) (1,2,0) (1,-1,1)


..

S. Hart


similar situation: they do not know what is the choice of the other player; forinstance, they may make their choices simultaneously. 0

Example 1.3. See Figure 6: N = {1, 2, 3}; root = a; po= {d}; pi

= {a, e, f};

V: = {a}; V~ = {e, f}; p2= vi = {b, c};

p3= V~ = {g}; payoff vectors, the

probability distribution at d, and the branches' correspondences [by (v) (a)] areall written on the tree. Note that at his second information set V;, player 1does not recall what his choice was at V:; so player 1 consists of two agents(one for each information set), who do not communicate during the play. 0

2. Pure strategies

Let i := {ViI' V~, . . . , V~(i)} be the set of information sets of player i; fromnow on we will simplify notation by using VI E l' to denote a generic elementof i. For each information set Vi of player i, let v ==v(Vi) be the number ofbranches going out of each node in Vi; number these branches from 1 throughv such that the one-to-one correspondence between the sets of outgoingbranches of the different nodes of Vi is preserved. Thus, let qvi):={1, 2, . . . , v(Vi)} be the set of choices available to player i at any node in Vi.

A pure strategy Si of player i is a function

si:I'--.?{1,2,...},

such that

SI(VI) E quI) for all Vi E i .

That is, Si specifies for every information set Vi E i of player i, a choice /( Vi)there. Let Si denote the set of pure strategies of player i, i.e.,Si:= I1UiEliqui). Let S:= SI x S2 x... X Sn be the set of n-tuples (or pro-files) of pure strategies of the players.

For an n-tuple s = (s\ S2, . . . , sn) E S of pure strategies, the (expected)8payoff hi(s) to player i is defined by

hi(S):= L p,(t)gi(t) ,tEL

(2.1)

where, for each terminal node t E L(T), we denote by p.(t) the probability

"Recall Remark 1.1.

l

26 S. Hart

that the play of the game ends at t when the players use the strategiess\ i, . . . , sn. This probability is computed as follows. Let 1T==1T(t) be the(unique) path from the root to the terminal node t. If there exists a playeri E N and a node of i on 1Tat which Si specifies a branch different from the onealong 1T, then Ps(t): = O. Otherwise, Ps(t) equals the product of the prob-abilities, at all chance nodes on the path 1T, of choosing the branch which isalong 1T.The function 9 hi: 5 ~ ffi defined by (2.1) is called the payoff functionof player i.

Example 2.2. Consider again the game of Example 1.3. Player 1 has four purestrategies: (1,1), (1,2), (2,1) and (2,2), where UP jz> means that j\ ischosen at U: and j 2 is chosen at U;. Player 2 has two pure strategies: (1) and(2), and player 3 has three pure strategies: (1), (2) and (3). To see howpayoffs are computed, let s = (2,1), (2), (3»; then the terminal node q isreached, thus hl(s)=1, h2(s)=-1, and h3(s)=1. Next, let s'=(1,1), (1), (3»; then h(s')=(1/2)(2,O,O)+(1/6)(O,2,O)+(l/3)(O,2,3)=(1,1,1).0

3. Games in strategic form

A second basic way of describing a game is called the "strategic form" (alsoknown as "normal form" or "matrix form,,).10

An n-person game in strategic form r consists of the following:(i) A set N = {1, 2, . . . , n} of players.

(ii) For each player i E N, a finite set 5i of (pure) strategies. Let 5:= 51 x52 X . . . X 5n denote the set of n-tuples of pure strategies.

(iii) For each player i EN, a function hi: S ~ ~)t, called the payoff functionof player i.

In the previous section we showed how the strategic form may be derivedfrom the extensive form. Conversely, given a game in strategic form, one canalways construct an extensive form as follows. Starting with the root as thesingle node of player 1, there are 15\ Ibranches out of it, one for each strategys \ E 51 of player 1.11The 1511end-nodes of these branches are the nodes ofplayer 2, and they all form one information set. Each of these nodes has 1521branches out of it, one for each strategy S2E 52 of player 2. All these 1511.1521nodes form one information set of player 3. The construction of the tree is

"The real line is denoted ffi.IOWe prefer "strategic form" since it is more suggestive.liThe number of elements of a finite set A is denoted by IAI.

~

S2

Head Tail

Head 1, -1 -1, 1s'

Tail -1, 1 1, -1


Table 1

.continued in this manner: there are Isil branches - one for each strategy Si E Si

of player i-going out of every node of player i; the end-points of thesebranches are the nodes of player i + 1, and they all form one information set.The end-points of the branches out of the nodes of player n are the terminalnodes of the tree; 12 the payoff vector at such a terminal node t is defined as(hl(s), h2(s), . . - , hn(s)), where s ==s(t) is the n-tuple of strategies of theplayers that correspond, by our construction, to the branches along the pathfrom the root to t-

Example 3.1. Let N={1,2}; Sl=S2={Head, Tail}; the payoff functionsare given in Table 1, where each entry is hl(S\ i), h2(s\ i). Plainly, theabove construction yields precisely the extensive form of Example 1.2 ("match-ing pennies"). D

(0,2)

(a)

(1,1) (2,0) (0,2) (2,0)

(b)

(1,1)

2,0 2,0

0,2 ',I

(c)

Figure 7. Two games in extensive form, (a) and (b), with the same strategic form, (c).

'2There are ISI=ls'I-ls21-.. .-IS"I terminal nodes.

I

28 S. Hart

It is clear that, in general, there may be many extensive forms with the samestrategic form (up to "renaming" or "relabeling" of strategies). Such anexample is presented in Figure 7. Thus, the extensive form contains moreinformation about the game than the strategic form.

4. Mixed strategies

There are many situations in which a player's best behavior is to randomizewhen making his choice (recall, for instance, the game "matching pennies" ofExamples 1.2 and 3.1). This leads to the concept of a "mixed strategy".

We need the following notation. Given a finite set A, the set of allprobability distributions over A is denoted .1(A). That is, .1(A) is the (IAI- 1)-dimensional unit simplex

.1(A) := { x = (x(a))aEA: x(a) ;=:0 for all a E A and L x(a) = I}.aEA

The set of mixed strategies Xi of player i is defined as Xi: = .1(Si), where Si isthe set of pure strategies of player i. Thus, a mixed strategy Xi = (X(Si))siESi EXi of player i means that i chooses each pure strategy Si with probability Xi(Si).From now on we will identify a pure strategy / E Si with the correspondingunit vector in Xi.

Let X:= Xl x X2 X ... X Xn denote the set of n-tuples of mixed strategies.For every x = (x\ x2, . . . , xn) E X, the (expected)!3 payoff of player i is

Hi(x):= L x(s)hi(s) ,sES

where x(s):= I1jENXj(Sj) is the probability, under x, that the pure strategy

n-tuple s = (Sl, S2, . . . , sn) is played. We have thus defined a payoff functionHi:X~m. for player i. Note that r*:=(N;(Xi)iEN;(Hi)iEN) is an n-player(infinite)!4 game in strategic form, called the mixed extension of the originalgame r = (N; (Si)iEN; (hi)iEN).

If the game is given in extensive form, one obtains [from (2.1)] an equivalentexpression for Hi:

Hi(x) = L pAt)gi(t) ,tEL

(4.1)

where, for each terminal node t E L(T), we let pAt) be the probability that theterminal node t is reached under x; i.e., pAt):= EsES x(s)Ps(t).

13Again, recall Remark 1.1.l4The strategy spaces are infinite.

.


5. Equilibrium points

We come now to the basic solution concept for non-cooperative games.A (mixed) n-tuple of strategies x = (x , x2, . . . , xn) E X is an equilibrium

pointl5 if

. Hi(x) ~ Hi(x-i, l)

for all players i E N and all strategies / E Xi of player i, where-i. ( I i-I i+1 n )d h ( 1) I f

..x .= x,... , x , x ,..., x enotes ten - -tup eo strategIes, In x,of all the players except i. Thus x E X is an equilibrium whenever no player ican gain by changing his own strategy (from Xi to /), assuming that all theother players do not change their strategies. Note that the notion of equilib-rium point is based only on the strategic form of the game; various "refine-ments" of it may however depend on the additional data of the extensive form(see the chapters on 'strategic equilibrium' and 'conceptual foundations ofstrategic equilibrium' in a forthcoming volume of this Handbook for a com-prehensive coverage of this issue).

The main result is

Theorem 5.1 [Nash (1950, 1951)]. Every (finite) n-person game has anequilibrium point (in mixed strategies).

The proof of this theorem relies on a fixed-point theorem (e.g., Brouwer's orKakutani's).

6. Games of perfect information

This section deals with an important class of games for which equilibriumpoints in pure strategies always exist.

An n-person game r (in extensive form) is a game of perfect information ifall information sets are singletons, i.e., Ivil = 1 for each player i EN and eachinformation set Vi E i of i. Thus, in a game of perfect information, everyplayer, whenever called upon to make a choice, always knows exactly where heis in the game tree.

Examples of games of perfect information are Chess, Checkers, Backgam-mon (note that chance moves are allowed), Hex, Nim, and many others. Incontrast, Poker, Bridge, Kriegsspiel (a variant of Chess where each playerknows the position of his own pieces only) are games of imperfect information..

15Also referred to as "Nash equilibrium", "Cournot-Nash equilibrium", "non-cooperativeequilibrium", and "strategic equilibrium".

30 S. Hart

(Another distinction, between complete and incomplete information, is pre-sented and analyzed in the chapter on 'games of incomplete information' in aforthcoming volume of this Handbook.)

The historically first theorem of Game Theory deals with a game of perfectinformation.

Theorem 6.1 [Zermelo (1912)]. In Chess, either(i) White can force a win, or

(ii) Black can force a win, or(iii) both players can force at least a draw.

We say that a player can force an outcome if he has a strategy that makes thegame terminate in that outcome, no matter what his opponent does. Zermelo'sTheorem says that Chess is a so-called "determined" game: either there existsa pure strategy of one of the two players (White or Black) guaranteeing that hewill always win, or each one of the two has a strategy guaranteeing at least adraw. Unfortunately, we do not know which of the three alternatives is thecorrect one (note that, in principle, this question is decidable in finite time,since the game tree of Chess is finite).16

The proof of Zermelo's Theorem is by induction, in a class of "Chess-like"games;17.18 it is actually a special case of the following general result:

Theorem 6.2 [Kuhn (1953)]. Every (finite) n-person game of perfect informa-tion has an equilibrium point in pure strategies.

Proof. Assume by induction that the result is true for any game with less thanm nodes. Consider a game r of perfect information with m nodes, and Ie. r bethe root of the game tree T. Let vI> vz, . . . , VK denote the "sons" of r (i.e.,those nodes that are connected to r by a branch), and let TI> Tz,. . . , TK(respectively), be the (disjoint) subtrees of Tstarting at these nodes. Each suchTk corresponds to a game rk of perfect information (indeed, since r has perfectinformation, every information set is a singleton, thus completely included inone of the Tk'S); rk therefore possesses, by the induction hypothesis, anequilibrium point Sk = (S~)iEN in pure strategies. From these we construct apure equilibrium point s = (/)iEN for r, as follows. If r is a chance node, then sis just the "combination" (or "concatenation") of the Sk'S i.e., Si(V) = s~(v) for

16There are "Chess-like" games - for instance, Hex - where it can be proved that the first playercan force a win, but nonetheless a winning strategy is not known. Other (simpler) games - e.g.,Nim - have complete solutions (i.e., which player wins and by what strategy).

17For example, see Aumann (1989, pp. 1-4).18Zermelo's Theorem 6.1 has been extended to two-person, zero-sum games by von Neumann

and Morgenstern (1944, Section 15).


all nodes v of player i that belong to Tk' If r is a node of, say, player i, then, inaddition to the above "combination", we pue9

/(r) := argmax h~(Sk) ,l,,;;k,,;;K

i.e., player i chooses at his first node r a branch k that leads to a subgame rkwhere his equilibrium payoff is maximal. It is now straightforward to checkthat S is indeed a pure equilibrium point of r. 0

Remark 6.3. The above proof yields a construction of equilibrium points inpure strategies by "backwards induction", from the terminal nodes to theroot:2o at each node of a player, choose a branch which leads to a subtree withthe highest equilibrium payoff for that player;21 at each chance node, averagethe equilibrium payoffs of the subtrees. Note that the equilibrium pointsconstructed in this manner, when restricted to any subgame of the originalgame, yield equilibria in the subgame as well; such equilibria are called"(subgame) perfect". The reader is referred to the chapters on 'strategicequilibrium' and 'conceptual foundations of strategic equilibrium' in a forth-

u~

(1,-1,-2) (2,3,0) (1,0,2) (0,1,3) (3,1,0) (4,',2) (1,2,-1)

Figure 8. The game tree of Example 6.4 and the construction of the pure equilibrium point.

.19We write h~ for the payoff function of player i in the subgame rk, and "argmax" for a

maximizer (if not unique, pick one arbitrarily).2{)This is the standard procedure of "dynamic programming"."Note that some of these choices need not be unique, in which case there is more than one such

equilibrium.

32 S. Hart

coming volume of this Handbook for a discussion of these issues of backwardsinduction and perfection in relation to equilibrium points.

The following example illustrates the construction.

Example 6.4. See Figure 8: arrows indicate the choices forming the equilib-rium strategies; the numbers in each node are the equilibrium payoffs for thesubtree rooted at that node. The resulting equilibrium point is s = (2,2),(1,2), (2,2»), with payoffs h(s)=(4, 1,2). D

The reader is referred to Chapter 3 in this volume for the development ofthe topic of games of perfect information, in particular in infinite games.

7. Behavior strategies and perfect recall

A pure strategy of a player is a complete plan for his choices in all possiblecontingencies in the game (i.e., at all his information sets). A mixed strategymeans that the player chooses, before the beginning of the game, one suchcomprehensive plan at random (according to a certain probability distribution).An alternative method of randomization for the player is to make an in-dependent random choice at each one of his information sets. That is, ratherthan selecting, for every information set, one definite choice - as in a purestrategy - he specifies instead a probability distribution over the set of choicesthere; moreover, the choices at different information sets are (stochastically)independent. These randomization procedures are called behavior strategies.

A useful way of viewing the difference between mixed and behaviorstrategies is as follows. One can think of each pure strategy as a book ofinstructions, where for each of the player's information sets there is one pagewhich states what choice he should make at that information set. The player'sset of pure strategies is a library of such books. A mixed strategy is aprobability distribution on his library of books, so that, in playing according toa mixed strategy, the player chooses one book from his library by means of achance device having the prescribed probability distribution. A behaviorstrategy is a single book of a different sort. Although each page still refers to asingle information set of the player, it specifies a probability distribution overthe choices at that set, not a specific choice.

We will see below that a behavior strategy is essentially a (special kind of)mixed strategy. Moreover, when a player has what is called "perfect recall",the converse also holds: every mixed strategy is fully "equivalent" to abehavior strategy.


We define a behavior strategy b' of player i in the game r (in extensive form)as an element of

Bi:= IT .1(C(Vi»,ViEli

(7.1)

that is, bi = (bi(Vi»ViEli, where each bi(Vi) is a probability distribution overthe set C( Vi) of choices of player i at his information set Vi. We will writebi(Vi; c), rather than the more cumbersome (bi(Vi»(c), for the probability

that the choice of player i at Vi is c E C(Vi); thus ~CEC(Vi)bi(Vi; c) = 1 andbl(V'; c) ~O.

Note that the linear dimension of the space of behavior strategies Bi ofplayer i is ~i (v~ - 1), whereas that of the space of mixed strategies X' isTIi v~ - 1, where j ranges from 1 to k(i) = 11'1and v~ = IC(V~)I. Therefore B' isa much smaller set than X'.

Actually, the set Bi of behavior strategies of player i can be identified with asubset of the set Xi of mixed strategies of i. Indeed, given a behavior strategy,one may perform all the randomizations (for all information sets) before thegame starts, which yields a (random) pure strategy - i.e., a mixed strategy.Formally, the mixed strategy Xi corresponding to the behavior strategy bi E Bi isdefined by Xi = (Xi(Si».iESi, where

Xi(Si):= IT bi(Ui; Si(Ui» (7.2)V'E/'

for each pure strategy Si E 5i. Since bi( Vi; s'( Vi» is the probability, under hi,that player i chooses s'(Vi) at the information set Vi, it follows that Xi(Si) isprecisely the probability that all his (realized) choices are according to the purestrategy Si; in short, Xi(Si) is the probability, under hi, of using s'. The followinglemma is thus immediate.

Lemma 7.3. For any behavior strategy bi E Bi of player i, the corresponding Xi

given by (7.2) is a mixed strategy of i that is equivalent to hi.

We call the two strategies land Zi of player i equivalent if they yield the

same payoffs22 (to everyone) for any strategies of the other players, i.e.,Hi(l, X-i) = Hi(Zi, X-i) for all x-I and all j EN. Note that the argument givenabove shows that a stronger statement is actually true: for each terminal nodetEL, the probabilities that t is reached under (bi, X-i) and under (Xi, X-i) arethe same, for any X-i.

. 22We have defined the (expected) payoff functions HI for n-tuples of mixed strategies (seeSection 4). The definition may be trivially extended to behavior strategies as well: use (4.1) withthe probabilities Px(t) computed accordingly.

L

34 S. Hart

The difference between behavior and mixed strategies can thus be viewed asindependent vs. (possibly) correlated randomizations (at the various informa-tion sets). This may be also seen by comparing directly the two definitions: Bi

is a product of probability spaces [see (7.1)], whereas Xi is the probabilityspace on the product [i.e., .1(TIU;E/;qui))]. The following example is mostilluminating.

Example 7.4 [Kuhn (1953)]. Consider a two-player, zero-sum23 game inwhich player 1 consists of two people,24 Alice and her husband Bill, and player2 is a single person, Zeno. Two cards, one marked "High" and the other"Low", are dealt at random to Alice and Zeno. The person with the High cardreceives $1 from the person with the Low card, and then has the choice ofstopping or continuing the play. If the play continues, Bill, not knowing theoutcome of the deal, instructs Alice and Zeno either to exchange or to keeptheir cards. Again, the holder of the High card receives $1 from the holder ofthe Low card, and the game ends. See Figure 9 for the game tree (A = Alice,

2 0 -2 0


23A two-player game is a zero-sum game if hI + h2 = 0, i.e., what one player gains is what theother loses.

24With a joint bank account.


Table 2Strategic form of Example 7.4

(S, K)

(S,E)

(C,K)

(C,E)

(S)

~'1+~'(-1)=0

(C)

~'1+~'(-2)=-~

~'1+~'(-1)=0 ~ . 1+ ~ . (0) = ~

~'2+j'(-l)=j ~.2+ 1'(-2)=0

~'0+~'(-1)=-~ ~ . 0 + ~ . (0) = 0

B = Bill, Z = Zeno; S = Stop, C = Continue, K = Keep and E = Exchange;payoffs at the terminal nodes are those paid by player 2 to player 1),

The strategic form of this game is given in Table 2. Note that the strategies

<S, K) and <C, E) of player 1 are strictly dominated (by

<C, K) and <s, E),

respectively). Eliminating them yields the reduced strategic form of Table 3.It is now easy to see that the unique optimal (mixed) strategies of the players

are (0,1/2,1/2,0) and (1/2,1/2), respectively;25 the value of the game is 1/4.Thus, in particular, player 1 can guarantee that his expected payoff will be atleast 1/4, regardless of what player 2 will do.

Suppose now that player 1 uses only behavior strategies. Let b 1= (b

I (U ~),b1(U ~» = «a, 1 - a), ({3, 1- {3» E B \ i.e., Alice chooses S with probabilitya and C with probability 1 - a, and Bill chooses K with probability {3and Ewith probability 1 - {3. [Note that the mixed strategy corresponding to b I is(a{3, a(l - {3), (1- a){3, (1 - a)(l - {3» - see (7.2).] Then player l's ex-pected payoff is26 (1- a)({3 - 1/2) if player 2 plays S, and a(l /2 - {3) ifplayer 2 plays C. So the maximum payoff that player 1 can guarantee whenrestricted to behavior strategies is

Table 3Reduced strategic form of Example 7.4

1

(S, E)

(C,K)

(S)

0

(C)

0

.25A mixed strategy of player 1 is written as the vector of probabilities for his pure strategies

(S, K), (S, E), (C, K), (C, E), in that order; for player 2, the order is (S), (C).26For example, the payoff if 2 plays S is computed as follows (1/2)'[a'I+(I-a)'

({3 . 2 + (1 - (3) . 0)] + (1/2) . (-1).

36 S. Hart

max [min{(I- a)(f3 -1/2), a(1/2- f3)}],O"'a.I3'" I

which equals 0, since either f3 - 112 or 1/2 - f3 is always:,;;; O. 0

Thus, behavior strategies for player 1 do a poorer job in this example thanmixed strategies: player 1 can guarantee 114 with the latter, but only 0 with theformer. Indeed, there is no behavior strategy corresponding to the uniqueoptimal mixed strategy Xl = (0, 1/2, 1/2, 0) of player 1, since Xl requires therandomizations at his two information sets to be fully correlated (rather thanindependent) .

The reason that behavior strategies are inadequate in Example 7.4 is thatplayer 1 consists of two agents who are not allowed to communicate during theplay. This implies that, in going from V~ to V~, player 1 "forgets" what heknew, namely the outcome of the initial draw. Therefore the player needs tocorrelate, before the game starts, the random choices of his agents at his twoinformation sets. Conversely, if a player always remembers what he knew aswell as what he chose at all his previous nodes - in which case we say that theplayer has "perfect recall" - then he has no need to correlate the choices at hisdifferent information sets: indeed, being at any information set uniquelydetermines what happened at all the previous ones.

Formally, given a game tree T and a node v of T, we will denote by T( v) thesubtree of T with root at v. For an information set V and a choice therec E C(V), we will write T(V; c) for the union of the trees T(w), where w isconnected to some node v E V by a branch labeled c [i.e., T(V; c) is the"remainder" of the game after the information set V has been reached, and thechoice c has been made there by the corresponding player]. We will say thatplayer i has perfect recall in the game r (in extensive form) if the followingcondition is satisfied.27 Let vj' V2 E

pi be nodes of player i, let Vii :3 VI andV ~ :3 v2 be the corresponding information sets of i, and assume that v2 E T(v I)(i.e., there exists a play of the game - a path - where v2 "comes after" VI);then there exists a unique choice c E C( Vii) such that V~ c T( Vii; c). A gamer in which every player has perfect recall is called a game of perfect recall. Notethat a player who is a single person has perfect recall;28 isolated agents are notneeded to play the game for him. This is the case for most parlor games (butnot for Bridge, when viewed as a two-player game, with each player consistingof two partners).

The condition in the definition of perfect recall can be separated into twoparts:

27The original definition of Kuhn (1953) is different but equivalent to the one presented here;the advantage of the latter is that it is stated (and may be checked) directly on the structure of thetree.

'"Provided he is not too absent-minded.


(i) "Player i recalls what he knew": V~ C T(V\), i.e., each node in V~ hasto be reachable from some node in V\. Otherwise, let v~ E V~\T(Vil)/9 then,when reaching the information set V~, player i does not recall whether the playwent through ViI (as in v2) or not (as in v~) - he forgot what he knew (when hewas at V\). If player i had perfect recall, then, at V~, he would be able todistinguish between v2 and v~, according to whether or not he has already beencalled upon to make a choice at V\; hence v2 and v~ would have lied indifferent information sets. In Example 7.4 above, player 1 does not haveperfect recall, since, if the play is: 'player 1 gets "High" and decides "Con-tinue" " then at V ~ he no longer knows what he knew at V ~ (namely, theoutcome of the draw).

(ii) "Player i recalls what he chose": V~ C T(V;; c) for a unique choice c atVii, Otherwise, let v~ E T(V\; c') n V~ for some other choice c' =1=c; then, at

V~, player i does not recall whether his own choice at V\ was c (as is the caseat v2) or c' (as is the case at v;) - he forgot what he chose (at V\). If player ihad perfect recall, then the nodes v2 and v; would be distinguished by thechoice he made at Vi\, and would thus lie in different information sets. InExample 1.3 (in Section 1), for instance, condition (i) is satisfied but (ii) is not:V ~ contains two nodes (e and f) that follow different choices (1 and 2,respectively) of player 1 at V:.

Example 7.5. In the game tree of Figure 10, assume that the root and thenodes at the third level do not belong to player 1 (it does not matter if they arechance nodes or personal nodes, and to which information sets they belong),and assume that all eight nodes at the fourth level are nodes of player 1. Ifplayer 1 has perfect recall, then condition (i) implies that {c, d, e, f} isseparated from {g, h, i, j} (i.e., there can be no information set of player 1containing nodes from both sets); condition (ii) separates {c, d} from {e, f},and {g, h} from {i, j}. The dashed lines in Figure 10 show thispartitioning. 0

Example 7.6. Modify Example 7.5 by putting the two nodes a and b at thesecond level in one information set (of player 1). Then perfect recall for player1 implies that {c, d, g, h} must be separated from {e, f, i, j} [condition(ii)]. 0

We come now to the main result of this section.

Theorem 7.7 [Kuhn (1953)]. Let r be a (finite) n-person game in which playeri has perfect recall. Then for each mixed strategy Xi E Xi of player i there exists acorresponding behavior strategy bi E Bi that is equivalent to xi.

'"A "\" denotes set subtraction.

38 S. Hart

c

Player I recallswhat he chose

Player I recallswhat he knew


Thus, having perfect recall is a sufficient3O condition for restricting a playerto behavior strategies instead of the (usually much) larger set of mixedstrategies.

Proof (outline). Given a pure strategy Si E Si and an information setVi E t, we will say that Vi is reachable under / if there exists a play of thegame that goes through Vi and is consistent with s\ i.e., there exists a path 'TTinthe tree that intersects Vi and, at every node of the player i on 'TT, the path

'TT

follows the choice dictated by Si. Given Xi E Xi we define the corresponding

behavior strategy bi E Bi as follows. For every information set Vi E 1\ let~nVi) be the probability that Vi is reachable under x\ i.e., I/(Vi) is the sum ofXi(Si) over all Si E Si under which Vi is reachable. Similarly, for each choicec E C(Vi) at Vi, let I/(Vi; c) be the probability, under Xi, that Vi is reachable

J()Kuhn (1953) shows that it is also necessary.


and the choice there is c [i.e., the sum of X'(S') over all / such that Vi isreachable and Si(Vi) = c]. Finally, put

bi(Vi; c):= (Vi; c)/I/(Vi) (7.8)

if the denominator is positive, and define bi( Vi) arbitrarily when gi( Vi)vanishes (it will not matter, since then Vi is never reached when i plays Xi).One may interpret bi( Vi) as the "observed (random) behavior" of player i atVi when he uses xi.

Let x -i be the strategies of the other players. We will show that

P(Xi. x-i)(t) = Pw. x-i)(t) (7.9)

for each terminal node tEL (these are the probabilities that the play ends atf). Fix t, and denote by 1T= 1T(t) the path from the root to t. Let Q' be theprobability that chance and all players except i always choose, at nodes on 1T,the branches along 1T; note that Q' depends on x -i but not on the choices ofplayer i. Similarly, let g (respectively, (3) denote the probability under Xi(respectively, hi) that all the choices of player i on 1Tare along 1T.Then (7.9)becomes Q'. g= Q'

. {3,and it suffices to show that g = {3when Q'> 0 (i.e., whent is reachable).

Let v] and V2 be two consecutive nodes of i along 1T(i.e., there are no othernodes of i between them), and let Vi] :3 VI and V~:3 V2 be the correspondinginformation sets; let c be the choice at v] along 1T.Player i has perfect recall,therefore V~ C T( V';; c), implying that V~ is reachable if and only if Vi] isreachable and the choice there is c. Hence gi(V~) = gi(Vi]; c); or, by (7.8), the

denominator of bi(V~;.) equals the numerator of bi(Vi]; c). To compute {3wehave to multiply the probabilities, under hi, of all the choices of i along 1T.Wethus obtain a telescoping product that simplifies to (3 = gi(V~; cm), where cm isthe choice of i along 1Tat his last information set V~ on 1T.Now gi(V~; cm) isthe probability, under Xi, that V~ is reachable and the choice there is Cm' or,equivalently (again by perfect recall, using induction), the probability that allthe m choices of i on 1Tare of the branches along 1T; but this probability isprecisely g. Therefore {3= g, and the proof is completed. 0

Theorem 7.7 has been generalized to infinite games by Aumann (1964). Asis pointed out there (p. 630), the extension of Kuhn's proof to the case wherethe length of the game as well as the number of choices at all information setsare at most countable poses no problems; the difficulties arise when there areuncountably many choices at some information set(s). In addition, games inwhich "time" is continuous pose special problems of their own, and do noteasily fit into the framework of this chapter (see Chapter 3 in this volume, and

40 S. Hart

chapters on 'two-player games', 'differential games' and 'economic applicationsof differential games' in forthcoming volumes of this Handbook for someexamples).

References

Aumann. R.J. (1964) 'Mixed and behavior strategies in infinite extensive games', in: M. Dresher,L.S. Shapley and A.W. Tucker, eds., Advances in game theory, Annals of Mathematics Studies52. Princeton: Princeton University Press, pp. 627-650.

Aumann, R.J. (1989) Lectures on game theory. Boulder: Westview Press.Kuhn, H.W. (1953) 'Extensive games and the problem of information', in: H.W. Kuhn and A.W.

Tucker, eds., Contributions to the theory of games, Vol. II, Annals of Mathematics Studies 28.Princeton: Princeton University Press, pp. 193-216.

Nash, J.F. (1950) 'Equilibrium points in n-person games', Proceedings of the National Academy ofSciences, 36: 48-49.

Nash, J.F. (1951) 'Non-cooperative games', Annals of Mathematics, 54: 286-295von Neumann, J. (1928) 'Zur Theorie der Gesellschaftsspiele', Mathematische Annalen, 100:

295-320. English translation: 'On the theory of games of strategy', in: A.W. Tucker and R.D.Luce, eds., Contributions to the theory of games, Vol. IV, Annals of Mathematics Studies 40.Princeton: Princeton University Press.

von Neumann, J. and O. Morgenstern (1944) Theory of games and economic behavior. Princeton:Princeton University Press.

Zermelo, E. (1912) 'Uber eine Anwendung der Mengenlehre auf die Theorie des Schachspiels',Proceedings of the Fifth International Congress of Mathematicians, Vol. II, pp. 501-504.

Documents

GAMES INEXTENSIVE AND STRATEGIC FORMS · Chapter2 GAMES INEXTENSIVE AND STRATEGIC FORMS SERGIU HART' TheHebrew University ofJerusalem Contents O. Introduction 1. Games inextensive