NZOR volume 6 number 1 January 1978

STOCHASTIC MODELS FOR COMPETITIVE ADVERTISING*

W, D. Cook

Faculty of Administrative Studies

York University

Toronto, Ontario

Summary

In this p aper some stoch a s t i c game models are d e v e l o p ­ed for determ i n i n g optimal ad v e r t i s i n g strategies for a firm with one or more c o m p e t i t o r s . It is, however, n e c e s s ­ary to modify the existing theory on m u l t i s t a g e games to account for budget c o n s t r a i n t s . In so d o i n g , it can be shown that for the infinite stage case, a behaviour s t r a t ­egy exists for each market state in each of the adv e r t i s i n g p e r i o d s . In a d d i t i o n , the models take significant account of past purcha s i n g b e h a v i o u r , wh i c h is an important factor that is not i n c o r p o r a t e d in the existing M a r k o v i a n models. Both finite and infinite stage models are d i s c u s s e d and solution proce d u r e s with i l lustrative examples are p r e s e n t ­ed.

1. Introduction

The p recise influence of adv e r t i s i n g dollars on the

public is extremely difficult to measure. If their effect

were b etter understood, met h o d s for reducing advert i s i n g

e x p enditure wo u l d p r o b a b l y be found. By its very nature,

the a d vertising function lias forced adv e r t i s i n g mana g e r s

to operate intuitively and subjectively. Recently, h o w ­

ever, d e cision m aking has begun to take a more rational

form, and the importance of long range p l a n n i n g and m a r k e t ­

ing information systems are n ow being recognized. M a r k o v i a n

models have been u sed to describe n o n - c o m p e t i t i v e m u l t i ­

stage advert i s i n g processes. Linear Pr o g r a m m i n g techniques

have been a p plied to advert i s i n g budget al l o c a t i o n problems,

and various "linear" learning models have b een developed.

B a yesian approaches appear to fit m a n y of the ev e r g r o w i n g

*Manuscript rece i v e d Oct. 1976; r evised M a r c h 1977

36

needs of m a r k e t i n g and adv e r t i s i n g management. S t ructuring

the p r o b l e m in B ayesian terms focuses attention on s i g n i f i ­

cant aspects, p i npoints inf o r m a t i o n needs, and provides a

m e d i u m of c o m m u n i c a t i o n among v a rious individuals involved

in a p a r t i c u l a r situation.

This pa p e r is conce r n e d w ith the development of m u l t i ­

stage (stochastic) game models for determ i n i n g optimal a d ­

v e r t i s i n g strategies for a firm w ith one or more c o m p e t i t ­

ors. By v i e w i n g the a d v e r tising p r o b l e m in this m anner it

is p o ssible to o btain the mu l t i s t a g e effect inherent in the

exis t i n g M a r k o v i a n models, w hile at the same time making

a llowance for c o m p e t i t i o n b e t w e e n firms \^hich are marke t i n g

a similar product. In addition, this approach takes some

account of b uying history, i.e., past buying beha v i o u r of

customers is brought to bear on current prob a b i l i t i e s of

e n tering the various market states. It is this very aspect

of ac c o u n t i n g for p u r c h a s i n g history, one w h i c h is rather

crucial in det e r m i n i n g an advert i s i n g strategy, w h i c h has

not been inc o r p o r a t e d in the M a r k o v models.

Before p r o c e e d i n g to develop these models, let us

b r i e f l y survey some of the r elated literature. Two major

p roblems w h i c h arise in the adv e r t i s i n g industry and for

w h i c h m a t h e m a t i c a l m odels have been developed, are (1 )

br a n d switc h i n g b e h a v i o u r by the c o nsumer and (2 ) media

selec t i o n by the advertiser. A first order Mar k o v chain

a p p r o a c h has been m ost commonly used in brand switching

models, since such an approach assumes that switching b e ­

h a v i o u r is a function of the p resent state only, and not of

past b uying behaviour. Such an assump t i o n leads to m a t h e ­

m a tical models w i t h easily expr e s s e d properties. Perhaps

the p r incipal reason for a dopting the Marko v i a n approach is

that the theory is well known and m a t h e m a t i c a l l y u n c o m p l i ­

cated. In the M a r k o v model d e v e l o p e d by K u e h n [ 10], it is

ass u m e d that the trans i t i o n prob a b i l i t i e s are the same for

all customers, remain constant from one purchase occasion

to the next and that the frequency and q uality of purchase

37

is the same for all customers. In this p a r t i c u l a r instance

the author is able to show that the model, in its aggregate

form, is equivalent to a model in w h i c h present sales are

affected by all p r evious advertising. This is, in general,

not true of most of the M a r k o v i a n m o dels which, to the c o n ­

trary, do not take buying h i story into consideration. M a r ­

kov chain models contain a num b e r of shortco m i n g s as p o i n t ­

ed out by Hern i t e r and H o w a r d ! 8 ] and by Ehrenberg[ 3].

In addition to the M a r k o v i a n approach, v a rious "linear

learning mod e l s " have been deve l o p e d in w h i c h the transi t i o n

probabi l i t i e s vary with p u r c h a s i n g behaviour. In fact it is

such a model that K u e h n l 10] shows is e q uivalent to his M a r ­

kov model. Such models take account of ad v e r t i s i n g exposure

in w hich each brand being m a r k e t e d has its switchable pool

whose members are likely to change brands if e xposed to a d ­

vertising. These learning models represent some of the more

s o p h i s t i c a t e d m a t h e m a t i c a l approaches to b rand switching.

Linear Pro g r a m m i n g m odels have b een found to be s i g n i ­

ficantly e f fective in solving the m e d i a selec t i o n problem.

There are many advantages to this approach. The technique

is simple to understand, it can be app l i e d to prob l e m s i n ­

volving a v ariety of media, it forces m a n a g e m e n t to make

precise definitions of markets to be reached, and can be

used by advertisers and agencies of any size. There are, at

the same time, severe limitations to the L.P. approach. S o ­

lutions are a r rived at w i thout c o n s i d e r a t i o n of audience

duplication, solutions are n o ninteger, and m a n y qualit a t i v e

factors have to be quantified. In addition, it is difficult

to account for c o m p e t i t i o n from competitors. For a c o m p r e ­

hensive analysis of L.P. models in advertising, see [4].

Enis [5] suggests that B a y e s i a n analysis may be a u s e ­

ful a pproach to the alloc a t i o n of adv e r t i s i n g budgets. B a y e ­

sian analysis incorporates subjective inform a t i o n in a l o g i ­

cal consistent manner, and focuses on econ o m i c c o nsequences

w h i c h are easily c o m m u n i c a t e d to top management. Enis p r o ­

poses a step by step proce d u r e for combi n i n g histo r i c a l data

38

in a logical m a n n e r and in so doing to arrive at an a c c e p t ­

able a d v e r t i s i n g strategy.

Little account has been taken of competing firms' a c ­

tion in the m a t h e m a t i c a l m odels in advertising. F r i e d m a n [ 6 ]

uses a game t heory a pproach to c o m p e t i t i v e l y spreading a d ­

v e r t i s i n g funds over different m a r k e t i n g localities. Mills

[ 1 2 ] studies various " a d v e r t i s i n g share" models in which one

company's m a r k e t share is its advert i s i n g expenditures d i v ­

ided by that of the total industry. Bell [ 1] analyzes an

adv e r t i s i n g c a m p a i g n from a game theoretic a p proach (a d i f f ­

erential game as in the F riedman m o d e l ) , in w h i c h there are

two c o m p e t i n g firms. He develops a model for obtaining an

optimal b a l a n c e in advert i s i n g al l o c a t i o n over N time p e r i ­

ods .

For f urther references see [ 2] , [13] , [14] and [15] .

2. An Infinite Stage A d v e r t i s i n g Model

A s t o c h a s t i c (multistage) game r = ( r ,..... T„) is a1 K

t w o - p e r s o n game which, at each m ove or stage t = 1,2,3,...,

is in one of a finite n u m b e r of states k = In each

state k there are a finite n u m b e r of alternatives or pure

strategies a vailable to each of the two players. In the a d ­

v e r t i s i n g context these pure strategies w ould be the various

dollar a l lotments that the firm in ques t i o n w ould be c o n s i d ­

ering as poss i b l e amounts to spend. If on a pa r t i c u l a r move

the game is in state k, and p l a y e r 1 chooses his pure

st rategy and pl a y e r 2 his j *"*1 pure strategy, the p ayoff fromJs

p l a y e r 1 to pl a y e r 2 is a... The choice of strategies i and' vj

j by the firms determines a set of t r ansition probabilitieskz k I

{p..}, where p.. denotes the conditional p r o b a b i l i t y that'l' J 0

the game will be in state 1 on the next move given that it

is n o w in state k and that pure strategies i and j are used

by the players.

For p u rposes of the model d e v e l o p e d in this paper, it

is a ssumed that per f e c t i n formation is available regardingk I kt

{p..} and {a..}. That is, each p layer knows the values of^ 7. 1 'l . ' l x '

39

k I k Ithe p.. and a.. prior to each move. The case in wh i c h the

i dtr ansition prob a b i l i t i e s are k nown only p a r t i a l l y and the

payoffs are random v a riables is dealt w i t h in an appendix

to this paper.

To put the advert i s i n g model in the pro p e r per s p e c t i v e

we pro c e e d as follows. A ssume that two m a j o r competitors,

w hich shall be referred to as firm 1 and firm 2 , are p r o d u c ­

ing a similar p roduce and that each spends a c ertain amount

on advertising in each of the P time periods P = 1,2,..., tt

during the year. tt might, for example, take the value 4 if

ad vertising were done on a macro basis 4 times a year. (In

case more than two firms are involved the p r o b l e m can be

analyzed from firm l's v iewpoint by grouping all other c o m ­

petitors t ogether and dealing w ith them as a single c o m p e t ­

itor.) Naturally, if the prod u c t s are to be m a r k e t e d over

a p eriod of years, it becomes n e c e s s a r y to dis t i n g u i s h p e r i ­

od P this y e a r from the c o r r e s p o n d i n g p e r i o d in any s u b s e ­

quent year. For this p u rpose let t = 1,2,... represent all

time periods or stages in the future w h e n advert i s i n g e x ­

pendi t u r e s will be made by the firms. It is a ssumed that at

the b e ginning of a stage each of the firms makes a decision

as to the a d vertising exp e n d i t u r e for that period, and then

at the end of the period, observes (1 ) the aggregate sales

level (for that period) and (2 ) the total ma r k e t share it

captured. Using these o b s ervations, each then decides upon

an a l location for the next period.

This a d vertising p rocedure thus constitutes a m u l t i ­

stage process at each stage t of w h i c h the s ystem can be in

any one of a n umber of states. In fact, for the sake of

simplicity, assume that at the b e g i n n i n g of pe r i o d P, the

system is in one of a finite num b e r of states k(P) =

1 , 2,. . . , K ( P ) . k(P) is simply an index r e p r e s e n t i n g the

state of the m arket at the outset of p e r i o d P. It shall,

in the p r esent context, stipulate both the level °f

total sales at the end of pe r i o d P-l and the fraction

of Qfc(p) atta i n e d by firm 1 during that period. It may,

40

w it h o u t loss of generality, be assumed that in state k(P)

each of the firms has a finite set of possible advertising

levels. Specifically, in state k(P) it shall be assumed

that firms 1 and 2 must select from among a set of possible

a d v e r t i s i n g levels }M.k(.P) and {q^/P }^k(P) . For the% 1 —1 J J — i

sake of n o t a t i o n a l convenience, a state will, henceforth,

often be d e n o t e d by k instead of k(P) w i t h the u n d e r s t a n d ­

ing that we are d i scussing a state in an arbitrary period P.

T r a n s i t i o n P r o b a b i l i t i e s :

One of the most serious drawbacks of a n umber of M a r k ­

ovian models is that no account is taken of buying history,

i.e., it is a s sumed that cons u m e r b uying habits in the next

p e r i o d depend only on that p e riod's advertising and not on

any previous p r eferences the cus t o m e r may have had. This,

of course, is a critical r e striction in most instances. In

reality, all past a d v ertising exp e n d i t u r e s have a crucial

effect on future buying habits.

In o rder to account for this past his t o r y aspect letk7

p . . be the condit i o n a l p r o b a b i l i t y that the system will be3

in state l(P+l) on the next move given that it is now in

state k(P) and given that firms 1 and 2 are selecting ad- k k

verti s i n g levels r. and q. respectively. In assigning val-kl ^ ^

ues to the {p. .} a num b e r of factors must be taken into con-

sideration.

First of all, if a firm's m arket share in the previous

p e r i o d was low, then it is likely that its share will remain

relati v e l y low in the present period. This will be e s p e ­

cially true in cases where the two firms' advertising e x ­

pe n d i t u r e s for p e r i o d P are the same or n e arly the same. To

capture this effect of a firm's m arket share being strongly

dependent upon its share in the previous per i o d it is ne-k I

c essary to define in such a ma n n e r that the p r o b a b i l ­

ity of going from a cer t a i n m arket share in one pe r i o d to a

d r a s t i c a l l y different m a rket share in the next p eriod is

r e l a t i v e l y small.

41

Second, the d e cision of a cust o m e r to switch brands d e ­

pends on a n umber of factors: R, the rate of decay of brand

loyalty; E , the p r o b a b i l i t y of past customers c o ntinuing

w ith the product; E , the p e r c e n t a g e of customers who will

buy a pa r t i c u l a r b rand in pe r i o d P, given that they bought

that brand in pe r i o d P-l, and a host of other factors. N a ­

turally, many of these determi n a n t s are difficult if not

impossible to measure accurately. However, assuming that

estimates of these factors can be obtained, the {p^}.) can^0

then be e x pressed in terms of them, p r o v i d e d that a f u n c ­

tional r e lationship w hich adequa t e l y represents the actual

s ituation can be determined.

Finally, since the state k(P) is a function of totalkl

sales as well as m a rket share, the (p..) can be def i n e d so

as to reflect seasonal changes in sales volume. For example,

as the Christmas season approaches, the p r o b a b i l i t i e s of g o ­

ing to higher and h igher sales volumes in the next p eriod

should be increasing. Hence, the model is reaso n a b l y s e n s ­

itive to cyclic affects. Thus, by a p p r o p r i a t e l y defining

{p..}, it is possible to make future sales a function not ^ J .

only of the present a d v ertising level, but also a f unction

of all previous adv e r t i s i n g levels.

Termin a t i o n States

The life of a produce depends on both total sales r e ­

sponse and the firm's market share. The deci s i o n on the

part of a firm to either w i t h d r a w its p roduce from the m a r k ­

et or make some m ajor al t e r a t i o n in its style or quality,

will be influenced largely by these two factors. The c r i ­

terion for te r m i n a t i o n will depend, of course, on the p o l i ­

cy of the p a r t i c u l a r firm in question. In general, it may

be assumed that w h e n some state k(P) is r e ached in which

both total sales and a firm's market share are so low as to

deem the produce o b solete or no longer appea l i n g to the

public, then the firm will w i t h d r a w its pro d u c t from the

market. W hether it is firm 1 or firm 2 that w i t h d r a w s its

42

product, this p a r t i c u l a r game w o u l d be co n s i d e r e d terminated.

Naturally, the t e r m i n a t i o n criterion in any given peri-

'1 *some o ther p e r i o d P2 , say the m o n t h of November. Hence, it

is n e c e s s a r y to allow for a nu m b e r of termination states -K ( P )

at least one for each period. If {k(P) (pj-;? denotes the

n o n - t e r m i n a t i o n states for p e r i o d P and the pr o b a b i l i t y of

e n t e r i n g a t e r m i n a t i o n state u pon leaving k(P) is denoted by

s ^ (P \ it follows that:

K T ’ * s k.‘.p > - il(P+l)=l

k IOr, u sing k instead of k(P) and letting p^ . = 0 for all

I I { 1 , . . . ,K(P+1 )} we have

n I kl ^ k ,(2.!) £ P in- + b .. = 1

1=1 z;} ^

w here K is the total num b e r of state KP=1

I n troducing the restri c t i o n s.. > 0 for all i,o,k, it

follows that the adv e r t i s i n g p r ocess will terminate with

p r o b a b i l i t y 1 in a finite nu m b e r of stages. For a p roof of

this see [ 16] .

Budget Constr a i n t s

Assume that in any state k a d v e r tising budgets B and v 1

B2 are avail a b l e for firms 1 and 2 respectively. The actual

sizes of these may, of course, not be known in advance, sok k

that B and B can be i n terpreted as the e x pected or esti-k

m a t e d budgets. It is r e c o g n i z e d that iB^} will change from

y ear to y ear and will, in general, show an u p ward trend.& "V

However, similar up w a r d trends ate inherent in {r .} , {<?.}"£• tJ

and {Q as well. M a k i n g the assump t i o n that the p e rcentage

growth rates of all components are relatively the same, then

all future budgets can be r e placed by their discou n t e d va-

lues and it may therefore be assumed that iB^} remain c o n ­

stant over time. This e s s e n tially stipulates that the

4 3

strategy spaces for the firms will not change over time.

Since the expe c t e d budget available to firm i w h e n e v e rk

the system is in state k is B ., then firm i must ensure 1 % * that its e xpected e x penditure does not exceed this level.

Hence, if firm 1 and firm 2 select their advert i s i n g levelsk k k

and {<7 0> a c cording to the m i x e d strategies X =

(xktx * 3 ...j X *^) and Y k = ( y * , y ...,y J ), respectively,

where x ^ is the p r o b a b i l i t y for firm 1 selecting s trategy iv k

in state k and y. is the p r o b a b i l i t y of firm 2 selecting3

strategy j in state k, then it is requ i r e d that the f o l l o w ­

ing conditions hold:

Mv MK V If k k k

(2.2) Z x. r. < J, w here Z x . = 1 ; x . > 0 V i, k1=1 1=1

N k j, k k N k k k(2.3) Z y . q ■ < B* w here Z y . = 1; y. > 0 V j, k

3=1 J J “ 3 = 1 0 J “

k k k k Let X and $ denote the sets of all vectors X and Y s a t ­

isfying (2.2) and (2.3) respectively. For n o tational c o n ­

venience Z Z and Z shall be u sed h e n c e f o r t h in p lace of i 3 I

M k N k KZ , Z and Z , respectively.

i=l j=i i=i

Stochastic Games

V iewing this m u l t i s t a g e process in terms of two-p e r s o n

zero-sum games, the immediate e x p e c t e d return from p e r i o d P

in terms of total sales volume is

(2 -4 > *1j ■ j r fj a i Q i

By virtue of the m u l t i s t a g e char a c t e r of the process

d e scribed above, it follows that such a p rocess m a y be v i e w ­

ed as a stochastic game T = (T; , , . . . ,r ) wh o s e ve c t o r v a ­

lue V = (vJtv 2, . ..,vK ) is the solu t i o n to

If V I(2.5) v k = V alue (a + Z pjj V , k = 1, 2, . . . ,K

44

w here "Value" represents the max - m i n value of the two-personk kl

z e r o - s u m game w hose p a y o f f m a t r i x is fa. . + E p. .v7). The

full set of pure and m i x e d strategies in such a game is

n a t u r a l l y c u mbersone since tec h n i c a l l y speaking there should

be a m i x e d s t rategy c o r r e s p o n d i n g to each state k and each

stage t. However, it will turn out to be n e c e s s a r y to i n t r o ­

duce a n o t a t i o n only for c ertain s t ationary strategies, i.e.,

those strategies w h i c h p r escribe for a p layer the same p r o b ­

abilities for his choices every time the same p o sition is

rea c h e d by w h a t e v e r route. Such s t ationary strategies can

be rep r e s e n t e d by K-tuples of p r o b a b i l i t y distribution

X = (x1 ,x2, . . . ,XK ) and Y = (Y 1 , Y 2, . . ., YK) .In the absence of budget constraints, the following

theorems for st o c h a s t i c games can be proved.

Th e o r e m 2 . 1 : There exists a unique solution V =

(v j , v 23 • • • , to the s ystem of equations (2.5) which r e ­

pres e n t s the v e c t o r value of the game r.

Th e o r e m 2 . 2 : There exists a pair of s t ationary strategies

X and Y such that X^ and Y ^ are optimal for firms 1 and 2,

respectively, in every subgame b e l o n g i n g to F.

Proofs of these results are found in [16].

The n o n l i n e a r s ystem of equations (2.5) can be solved

by means of an iterative technique w h i c h employs a c o n t r a c ­

tion map p i n g w hose unique fixed point is the value of the

game. This map p i n g is a p plied recurs i v e l y from a selected

s t arting point, and each iteration of the m apping is p e r ­

formed by solving a set of linear p r o g r a m m i n g problems.

T r u n c a t i o n of the r e cursive technique yields c-effective

(near optimal) strategies for T in the sense that these

s t rategies give rise to scalars w hich are w ithin e of

the game values v^. In fact, it is p o ssible to determine,

in advance, the effect on the total expe c t e d payoffs when

such strategies are used. For a p r o o f of these results see

[ 16] .

45

In order to define the a p propriate con t r a c t i o n m appingftconsider the K - d i m e n s i o n a 1 space R w i t h n orm

ftIYH = Max y, where y = (y ,y , ... , y e R .

l<k<K 1ft ft

Let the map p i n g T : R -* R be d efined by

Ta = 6 = (6 j , B o , •••,e^) where = Value

, k v kl ,(a . . + Z p. . a J .

i j £ I

In the p roof of theorem 2.1, Shapley [16] shows that

U Ta - Ta^ W (1-s) II a'' a Z II for ,a^ e R*

where

, k ,s = m m {s . .}

Since 0 < _ l - s < l , T i s a contra c t i o n m a p p i n g and t h e r e ­

fore has a unique fixed point V = (y^ ,v2 , ...,y^).

Next, c onsider the sequence {Kr } w h i c h is def i n e d r e ­

cursively for a given by v r + * = TVr . Hence

( \ r +1 , kl , v kl r ,(2.6) v k = Value (a.. + I p ^ . v ^

By the contra c t i o n prop e r t y of T, the sequence of {Vr } c o n ­

verges to V for any fixed V .

Setting V = 0, the sequence {V } can be c o mputed by

linear pro g r a m m i n g techniques. Indeed, the p r o g r a m in

the following c o llection is a linear p r o g r a m m i n g formul a t i o n

of the t w o-person z e ro-sum game whose p a y o f f m a t r i x is, k _ kl r . fa.. + Z p . .v 7 ) .

^ Z- ^ r»-/-7 V+l r*+l r*4- 7To determine K = (y^ ^y^ ) solve the if

linear p r o g r a m m i n g problems

46

subject to

(2-7) < i \ * ki ,aic + \ p ij V l>’ C' ‘ u 2 ..... "k

I X k. = 1, X k. > 0 Vi ,v

k = 1 2 K> > ■ • • >

, r + 2 a?w here = a^.

In this m a n n e r the sequence {yr } is generated.

When c o mputing the sequence {yr } it is desirable to

have a stopping c r i t e r i o n w h i c h insures a d esired a p p r o x i ­

m a t i o n of V. Given e, it can be shown that if r(e) is such

| v r(c)+l _ v r(z)\ < £ / 2 £or a U then vr ( e ) + l _ ^ | <

In fact, after a single i t eration an upper bound R on the

m a x i m u m n u m b e r of iterations r e quired to attain this p r e c i ­

sion can be determined, i.e., after K "2 is c omputed from V^

= 0, choose R such that

(1~s)R. • II v1 - V° II < e . s

It then follows that for some r(e) R, the stationaryk k

strategies {X } and {y } o b t a i n e d from (2.7) are e-optional

(e-effective) for the m u l t i s t a g e game.

C o n s t r a i n e d Stocha s t i c Games

In the p r e s e n c e of budget c o nstraints (2.6) becomes

r-, n-% r+1 ■ v v k , k , _ kl r , k(2.8) v , = m ax min E E x. (a.. + T. p - - v 1 ) y . i

X K zX Y e<J> J

w h i c h is e q uivalent to the linear p r o g r a m m i n g p roblem

zvr nk t ^ „k ra k + 2 =mSX % + V *

47

subject to

r ^ k r _ k , k , v kl r , .* "j Ik i Z. *i (aij * ? P ij l ’ 3 =

C2.9) Z X* r1. -J ^ 1

X

E = 1, x^ _> 0, £ = 1,2,... .i

It is w o r t h n oting that the budget c o nstraints can be set to

e q uality since they will be such at the o ptimum anyway.

If one proceeds as before by starting with V® = 0 and7*+1 *1* k. ^ T*

setting v^ + B^ then the same results as those

given earlier for u n c o n s t r a i n e d stochastic games follow.J Is 1 y* *

This is due to the fact that V a l u e (a.. + I p.. v~) is stillij i I

a c o ntraction mapping. The ques t i o n of f e asibility andv

b o undedness are of no concern here unless for some k, r > k k k

Bj or > B 2 ' Thus o btain an e-optimal advert i s i n g

strategy sequence, solve (2.9) r e c u r s i v e l y until

have been found for w h i c h max | uf - w f - "2 | < e/2. The set of ~k K

m ixed strategies {X )-k _ j w h i c h determine these values c o n ­

stitute the desired sequence, and the iterative p r o c e d u r e

can be terminated.

An E x a m p l e :

Let the year be d i vided into tt = 3 a d v e r t i s i n g p eriods

where, say, P = 1 is the p e r i o d Jan u a r y to April, P = 2 , the

p e r i o d May to Septe m b e r and P = 3 , the pe r i o d O c t o b e r to

December. Assume a total of A' = 8 states (2 in P = 1, 2 in

P = 2 and 4 in P = 3 ) , and let the budgets available to

firms 1 and 2 in the various states be given by B k - $225

(thousand), = $275 for k = 1,2,3,4; B k = B k = $350 for

k = 5,6; B k = $900 , B k = $800 for k = 7; and B k= B k = $1,050J Cj 1 O

for k = 8. Assume that each firm has 3 poss i b l e alternatives

in each state. In each of states k = 1 , 2 , 3 ,4 let the a l t e r ­

n atives (possible advert i s i n g allotments) be ($1 0 0 , $ 2 0 0 ,

$300); in k = 5,6 ($200, $300, $500); in k = 7 ($700, $800,

48

$1000); and in k = 8 ($900, $1000, $1200). At the end of

p e r i o d 3 (end of December) let the possible (sales volume,

m a r k e t share) pairs be (2000,1/5), (2000,3/5), (3000,1/5),

(3000 ,3/5). Denote these as states k = 1,2,3,4, r e s p e c t i v e ­

ly. At the end of p e r i o d 1 (end of April) assume (sales

volume, m a rket share) = (1000,1/3) or (1000,2/3). Denote

these as states k = 5,6. At the end of period 2 (end of

September) assume (sales, volume, market share) = (1500,2/5)

or (1500,3/5). Denote these as states k = 7,8. Assume that

the following tr a n s i t i o n p r o b a b i l i t i e s have been determined

15 (p . ■) -

1/3

1/4

1/2

1/3

2/3'

1/216

‘P i o’ =

'1/5

1/5

1/7

1/6

1/8

1/7

.1/5 1/4 1/3. .1/4 1/5 1/ 6 .

< * % >

1/7 1/6 1/5'

(p2.6.)

'2/3 1/3 1/5'

1/8 1/7 1/6 9 2/3 1/3 1/4

1/9 1/8 1/7. .2/3 1/2 1/3 .

U .) , (P -, 21. (p ■ ■)

10for 1 = 5

(P 5?.)

'1/2 2/3 3/4'. 58 (p . .) =

T'J

'1/4 1/5 1/6 '

= 1/3 2/3 2/3 y 1/3 1/4 1/5

.1/5 1/4 3/5. .1/2 1/3 1/4 .

'1/3 1/2 2/3'

(p ■ ■) =

'1/4 1/5 1/5 '. 67 . ( p ■ J = 1/4 1/3 1/2 y 1/3 1/4 1/5

1/5 1/4 l / 2 _ 1 / 2 1/3 1/4 _

'1/10 1/9 1/8

< r n > ~

'1/12 1/13 1/14'

r 71 i = 1/11 1/10 1/9 y 1/11 1/12 1/13

. /12 1/11 1/10 1/10 1/11 1/ 12.

/ 7 3 )

1/8 1/7 1/6. 74 .

tpij> -

"1/10 1/11 1/ 12'

= 1/9 1/8 1/7 1/9 1/10 1/11

1/10 1/9 1/8 1/7 1/8 1/9

49

(pfj) = (plj) for all Z, (pk}.) = (03 x 3 ) for all other k,l.

As an i n t erpretation of the above, cons i d e r (p ??) for10

example. The value p|^ = 1/8 means that if at the end of

December firm 1 observes that it is in state 2 (i.e., its

sales volume for the p e riod Oct o b e r through D e c e m b e r was

2000 units, wh i c h re p r e s e n t e d 3/5 of total sales for that

period), and if it allocates r 2 = $200,000 to advert i s i n g

while firm 2 allocates q 2 = $ 1 0 0 ,0 0 0 , there is a p r o b a b i l i t y

of 1/8 that firm 1 will find itself in state 5 at the end of

April, i.e., there is a 12.5% chance of its sales at the end

of April being 1000 units or 1/3 of the total sales for the

period Jan u a r y through April. There is a p 2^ = 2/3 = 66.7%

chance of firm 1 being in state 6 (i.e., h aving 2/3 of the

market) at the end of April. Hence there is a (100-12 ..5 -6 6 .7) %

= 20.8% chance that the m arket results at the end of April

will either abandon its product or make a m a j o r change in

design. In either case there is a s 2 = 20.8% chance of

t e r mination of the game a s s o c i a t e d w i t h state 2 and a l t e r ­

natives i = 2, j = 1 for firms 1 and 2 respectively.

Linear pro g r a m m i n g was used to find an e-optimal s e ­

quence of strategies. After 9 iterations IIV9 - V ll < 1/2

and the required e - e ffective s olution is (assuming e = 1)

V 9 = 5400 . 7 , v 92 = 6423. 8 , v 9r = 6179.4, v 96 = 5439.8 ,

v 9 = 4446.6 , v9 = 4452.1 , v 9 = , v9 = v9 .

The c o r r e s ponding m i x e d strategies for firm 1 are

x : = X2 = X3 = X 4 = (.375, 0, .625)

= (.557 ,.058 , . 365),

X6 = X8 = (.5, 0, .5) , Z 7 = (.33 , 0 , .67).

To interpret the e-optimal strategies, c o nsider

X* = (.375, 0, .625). When in state k = 1, firm 1 must

choose one of the three possible allocations ($1 0 0 , $ 2 0 0 ,

50

$3001. If m a n a g e m e n t selects from these r a ndomly according

to X , there is a 37.5% chance that the d e cision will be to

allo c a t e $100, w i t h a 62.5% chance of $300 being chosen.

3. A Finite Stage A d v e r t i s i n g Model

As is often the case, a firm may w ish to plan an a d v e r t ­

ising c ampaign to e xtend over a fixed time period. In a pre-

C h ristmas campaign, for example, each firm wo u l d want to a l l ­

ocate its adv e r t i s i n g funds on e ither a daily or weekly

basis, say, in such a w a y that total sales are maximized. In

such a finite h o r i z o n model, the nu m b e r of moves in the m u l ­

tistage p r o c e s s is a ssumed to be at most T . In this case the

strategies, p ayoffs and trans i t i o n p r o b a b i l i t i e s may be a s ­

s u m e d to be functions of the stage t. For this reason we

i ntroduce the n o t a t i o n X k (t), Y k (t), a k .(t)j p k ]-(t), M, andI'd 'Is J K.K

As in the infinite stage model we impose budget r e ­

s t rictions analogous to (2.2) and (2.3) and denote thek

c o r r e s p o n d i n g s trategy spaces for firm 1 and firm 2 by X (t)v

and $ (t) respectively.

In order to solve the finite stage game let v^(t) denote

the optimal exp e c t e d p a y o f f to be received by player 1 from

the r e maining T - t moves w h e n the games is in state k and

stage t , i.e.,

To determine given {v^(t)}, solve the set of K

linear p r o g r a m m i n g problems

(3.1)v k (T) = 0

v ^ ( t - 1 ) =

X k ( t)eXk (t) Y k ( t ) d k (t)

£ Z x k (t ) (a k. . (t) i J 1

+ Z p ^ l-(t) v 7 (t)) y k.(t)

+ B k y^(t) - max + B k Y ^ f ^

51

subject to

(3.2)

e x k.(t) = i, > o ,t V =

Set v^(t-l) = a^(t) +

7(0) = (y1 (0 ) , ...,y (0 )) is the v ector value of the game and

wh i c h are equivalent t „ j problems (3.2),

are optimal in move t of the stoch a s t i c game.

An E x a m p l e :

Let T = 5 and take all other data to be as d e f i n e d in

the example of Section 2. In this case note that calucla-

tions will be simpl i f i e d by virtue of the fact that the data

is not time dependent. Note that stages 1 and 4 c o r r e s p o n d

to states 1, 2, 3, 4; stages 2 and 5 to states 5 and 6 ; and

stage 3 to states 7 and 8 .

P r oceeding as d e s c r i b e d above the ve c t o r value of the

XJ (1) = X 2 (l) = X3 (1) = X4 (1) = (. 375 , 0 , .675),

r 5 (2) = (. 531 , . 133 , . 356) , X6 (2) = (. 5 , 0 , . 5),

/ ( 3 ) = (. 333 , 0 , .667) , X8 (3) = (. 5 , 0, .5),

X J (4) = * 2 (4) = X 3 (4) = * 4 (4) = (.375, 0, .675),

X 5 (S) = (.376, .299, .325), X 6 (5) = (.5, 0, .5) .

the optimal strategies X (t), Y (t) in the two-p e r s o n zero-k kl

sum games w ith p a y o f f m a trices (a..(t) + E v..(t) v ^ ( t ) ) }

game is V = (y^ (0) , v£ (0) , y^ (0) , y^(0)) = (2326 , 2740 , 2326 ,

2740). The optimal m i x e d strategies are

52

A c k n o w l e d g e m e n t s

This research was s u p p o r t e d in part u nder Ford F o u n d a ­tion Grant #24/221/01 and N.R.C. Grant A 8966.

R EFERE N C E S

[1] Bell, C.E., "The N Days of Christmas: A Model for C o m ­p e t i t i v e A d v e r t i s i n g Over an Intensive Campaign",Man. Sci. 14 (1968), 525-535.

[2] Deal, K.R., and S. Zionts, " S ingular Solutions of aD i f f e r e n t i a l Game Model for D e t e r mining Competitive A d v e r t i s i n g Policies", p r e s e n t e d at Fifth Annual CORS C onference, M ay (1973).

[3] Ehrenberg, A.S.C., "An A p praisal of M a r k o v Brand-S witc h i n g Models", Journal of M a r k e t i n g R esearch 2 (1965), 347-362.

[4] Engel, J.F., and M.R. Warshaw, "Alloc a t i n g A d v ertisingD ollars by L inear Programming", Journal o f A d v e r t i s ­ing R e s e a r c h Fo u n d a t i o n 4 (1964), 42.

[5] Enis, B.M., "Bay e s i a n A p p r o a c h to Ad Budgets", Journalof A d v e r t i s i n g R e s e a r c h 12 (1972), 13-19.

[ 6 ] Friedman, L., "Game Theory Mo d e l s in the Alloc a t i o n ofA d v e r t i s i n g E x p e n d i t u r e s”, O p erations Research 6 (1958) .

[7] Gillette, D. , "Stoch a s t i c Games with Zero Stop P r o b a b ­ilities", Ann. of Math. Studies 39 (1957), 179-187.

[ 8 ] Herniter, J.D., and R.A. Howard, " S tochastic MarketingModels", in Prog r e s s in Opera t i o n s R e s e a r c h , Vol. II, ed. by Hertz and Eddison, W i l e y New York (1964).

[9] Hoffman, A.J., and R.M. Karp, "On Non-Te r m i n a t i n gS t ochastic Games", Man. Sci. 12 (1966), 359-370.

[10] Kuehn, A . A . , "A Model for Budge t i n g Advertising", inMat h e m a t i c a l Models and Met h o d s in M a r k e t i n g , ed. by Bass et al, Irwin (1961).

[11] Maitra, A., and T. P a r t h a s a r a t h y , "On Stochastic Games",JOTA 5 (1970) , 289-300 .

[12] Mills, H.D., "A Study in Promotional Competition", inM a t h e m a t i c a l Models and Methods in M a r k e t i n g , ed. by Bass et al Irwin (1961).

53

[13] Naert, P.A., " O ptimizing Cons u m e r Adverti s i n g , I n t e r ­m e diary A d v e r t i s i n g and M arkups in a Vertical Market Structure", Man. Sci. 18 (1971), 90-101.

[14] Ray, M.L., and A.G. Sawyer, " B ehavioural M e a surementfor Marke t i n g Models: Estim a t i n g the Effects of A d ­v ertising R e pition for M e d i a Planning", Man. Sci. 18 (1971), 73-89.

[15] Sasieni, M.W., "Optimal A d v e r t i s i n g E x p enditure", M a n .Sci. 18 (1971) , 64-72 .

[16] Shapley, L.S., "St o c h a s t i c Games", Proc. Nat. Acad. Sci.39 (1953), 1095-1100.

[17] Takahashi, M . , " S tochastic Games with Infinitely ManyStrategies", Journal of S c i e n c e s , H i r o s h i m a U n i v e r s ­ity 26 (1962) , 125-137 .

APPENDIX

Partial Information in M u l t i s t a g e Game

kAssume that in state k the p a y o f f m a t r i x (a..) is a

'Z' 0random variable w h i c h can take on a finite nu m b e r of possible

values (a--(m)'), m = i , 2 ,. . . , 0.. As s u m e in addition that on-7* %J K

ly partial information is a v ailable r e garding the d i s t r i b u ­

tion q k= {qk (1 ) ,qk (2) , . . . ,qk (0 , ) ) T of {ak .) , and that thisK 1s J

information appears in the form of a finite sy s t e m of lineark k k

inequalities, E q < 0, on the components of q . Define

(A. 1) n k = {qk eR k \ £ q k (m)=l, E k k < 0, q k (m)>0 m ] , ym } m H “

k = 1,2 , . . . ,K.

In addition, assume that the trans i t i o n p r o b a b i l i t i e s1/ Ts 7 Is 9 Ts V

p..(m) = (p . .(m),p . .(m)} ...,p . .(m)) are only p a r t i a l l y speci-i'd I'd I'd k k

fied. Specifically, let E..(m)P..(m) < 0 be a finite s ystemT' J i'C 6=1 t,

of linear inequalities and assume only that the {P . .(m)} are

elements of the linear po l y h e d r a l set

l[k .(m) = {Pk. .(m) z R ^ \ l p k \(m) = 1 - s k . (m) , E k . (m ) P k . (m) < 0, ^3 13 1 i 13 13 13 13 = ’

(A.2) p ) lJ m ) > 0 VZ},1 3 = *

54

i 1,2,... , j 1 , 2 , • . . , N > k - 1 , 2 , . . . ,K,

m = 1 ,2, ...,6 ,

Note that there is a ve c t o r of t r ansition probabilitiesk k

P..(m) and a ter m i n a t i o n p r o b a b i l i t y s..(m) c o r responding to^ 3 k

each value of the ra n d o m variable a... For purposes of thek

f o llowing analysis the s . .(m) shall be a ssumed to be knownto

q u a n t i t i e s .

Since payoffs and trans i t i o n p r o b a b ilities of the game

are not co m p l e t e l y specified, it is nece s s a r y to m odify the

d e f i n i t i o n of the "Value" of such a game. Let a =V

( a . j a . ..... a j be some v ector in R and consider the two-k k

p e r s o n z e ro-sum game w i t h p a y o f f ma t r i x ( a . ■ + P ■ -a). Ifij vj

pl a y e r 1 selects the c r i t e r i o n of ma x i m i z i n g his guaranteed

pay off, the p r o b l e m wh i c h he must solve to determine hisA

optimal m i x e d s t rategy X is

k k k V alue (a.. + P.. a) = m ax min m in min X [Z(a..(m)

t3 k k k k mX K Y q (P

i j ( m ) )

+ P k .(m)a) q k ( m)]Yk^t7

Subject to

(A.3) X k e \ k = {Xk eR k \Zxk = 1 , x k > 0 V i}i

A * *- U k z R k \lyk, - 1, #‘ » 0 V j)3 0

q k z n k3 P k..(m) en k--(m).^ t3 'Z-J

To show that there exists an optimal vector V=(v ,. . ,v )1 K

satis fying

y , = Value (a k . + P k . V)k ------ 1 3

kWe first con s i d e r the case in w hich a., is deterministic.

13(A.3) then becomes

55

Value (ak . + P k .a) = max m in X [ (a k . + min^'7 vk k vk k ZJ vk ellfe

X eX e<i> p ij ij

P k. .a)]Yk13

Proofs of the following lemmas are s t r a i g h t f o r w a r d and,

hence, are omitted.

- k -~,kLemma A.l: Let (a..) and (a..) be two mxn matr i c e s d efined bv--------------- 1 3 1 3

a k. . = d k. . + min P k. .a and cJk . - d k. . + min P k. .a13 1 3 7, 1 0 1 0 1 3 p % t o

P. .en.. P. .ell..1 3 t 3 1-3 1 3

Then if 9 k. . = d k. . + h min P k. . a and clk. . = d k. . + min 1 3 13 P k..en.. 13 1 3 13 „k

13 13 i j i j

-k ~kThen if e.. d.. + a for all i.j.k and some scalaraa, it

is q is q

follows that Value (ck .) < V alue (dk .) + a13 13

Lemma A . 2: If {y „-)”= i and are any two finite real— — — — — — — — ~ls 1 ,-- 1 % 'Is — 1

sequences then Imin {5.} - m in (y .} < wax {|6.-y.|}.1 . 1 1 ' 1 1 '

1 1

With the exce p t i o n of the "min" o p e r a t o r inside the

Value function, theorems A.l and A . 2 are similar to theoremskl

1 and 2 in [16] for known {p..}. U t i l i z i n g lemmas A.l andTs{J

A. 2, the proofs follow in m u c h the same manner. For this

reason the details are omitted.

T he o r e m A.l: There exists a unique vector V =k k

v K ) satisfying = Value (a ij. + ^min^ p ic-a ^ k =

P i j Z 1 3w hich represents the lower value of the m u l t i s t a g e game

T = (TJ ,r2 , . . . ,r^) as v iewed by p l a y e r 1 .

T h e o r e m A . 2: There exists a sequence of s t a t i o n a r y strate-a L - V * L V

gies {3P anc* ^ ^k=l are optiroal for players 1

and 2 respectively in the sequence of games c o m p r i s i n g the

stochastic game T = ( F ^ , , . . . , , in the sense that

For the case ^ > 1 we prove the following theorems

w h i c h are analogous to those g iven above.

T h e o r e m A. 3 : There exists a un i q u e vector V = . ..,

vK ) s a tisfying

k k(A.4) v = max m i n m in X [ Z ( a . A m ) +

Yk v fc vk .k k _£ m lJ X ex Y e$ q eII

min (Pk. A m ) V ))qk (m)]Ykk k

P. Am)ell. A m )1 3 1 3

for k = l,2,...,K. V represents the v ector value of the

m u l t i s t a g e game (T1 ,T2 , . . . ,TK ) as viewed by player 1 .

P r o o f : Define a c o n t r a c t i o n map p i n g v£ analogous to that

in s ection 2. That is, let 7^ = 0 and let be defined

similar to u. in (A.4) but w ith V , X k ,Yk and q k replaced by

Vr ,Xkl> and q k r . L e tting {<7^} ^ d e n o t e the extreme points

of H k , it follows tha£ each q r en^ can be expressed as a c o n ­

vex c o m b i n a t i o n q k r =T. k n \ krq kr of these extreme points .Define H x=l t t r

a k . = Z a k. A m ) q k (m) and v kr. =Z( min (P k. A m ) V ) ) q k (m) .1 3 1 m 1 3 ^ t ^ J T m T, j , 1 3 T

m m P. A m ) e n . A m )It then follows that

(A. 5) = max min min X k r [Z(ak . + v kr. )\ k r ] Ykrk ,kr k „kr k ,kr - T

X EX Y e$ XT

Sub j e c t to E \kr = 1, X*r > 0 V t

T

Def i n i n g Wkr. = Xkl,y kr and Wkr = (Wkr.), (A. 5) is equivalent toT J T J T 3

57

(A.6 ) subject to E Akr = 1, \kr > 0 V TT T

T 1

y e <J>

..kr .kr kr W = A y . .

It is easily shown that the yj^ r = 0,1,2,... converge

to some value v ^ satis f y i n g (A.4) and rep r e s e n t i n g the v e c ­

tor value of T.

Tlieorem A . 4: There exists a sequence of st a t i o n a r y strate-~K K

gies {X {N ^fe=l are optimal for pl a y e r 1 and the

c o alition of nature and pl a y e r 2 r e s p e c t i v e l y in the sequence

of games comprising the m u l t i s t a g e game T.

a !/ a LProof: Let {X } W } be a p air of optimal m i x e d strategies in

-k kthe two-person zero-sum game w ith pa y o f f m atrix + v iji^

where v k. . = £ ( min (Pk. . (m) V)) q k (m) .m k k T

P . . (m) cII . . (m)ij

By arguments similar to those given in T h e o r e m 2 of [ 16] , it

can be shown that the optimal po l i c y at each stage t is to

emplov these m i x e d strategies w h e n e v e r the game is in state k.kl k

As in the case of known {p..} and d e t e r m i n i s t i c (a'..)> it

is possible only to obtain e-optimal m i x e d strategies. E s s ­

e ntially we pro c e e d as follows. In o rder to reduce (A.4) to

the more manage a b l e form [A.6 ), it is n e c e s s a r y to computek

the extreme points of the set n . A l t h o u g h met h o d s do exist

for doing this, for general p o l y h e d r a it can be a cumber s o m e

process. For certain special structures such as those a r i s ­

ing as a result of a partial o r dering on the c o mponents ofk

q , it is p ossible to cha r a c t e r i z e these extreme points. In

any event, given V and letting A denote the extreme points

of II , solve the sequence of linear p r o g r a m m i n g p roblems

m in P k. .(m) Vrij

(A. 7) Subject to P k. A m ) z i= 1, 2, . . .,M. ; j = l t 2t ...,N ■K K

171- 2 3 2 J . . • J 2 y • • . yK •

58

Using {A }, evaluate (a.. } and (u . . }, and solvet,7 T t,7 t ’t j -

'fer /cra

kva < E x k.(ak. .

- i V 1JTkr . . , 0

J = ^ • ...*k

kZx . . t

= 1 , x k. > 0X —

i = 13 23 . ..,Mk

k = 1,23 ...3K

Setv+l '•kr

V, = CL k

There exists a finite index r(e) such that |yj^e ^+ * -

I < e/2 for all k, in w h i c h case 1 ^ ^ * \ < e. The

s t a t i o n a r y strategies X k and w k , obt a i n e d when (A.8) is

solved w ith r = r(e), are e-optimal for the multi s t a g e game.

In the case of a finite stage game, t <_ T , with randomk k

a..(t) and p a r t i a l l y s p e c i f i e d P . . ( t , m ) 3 results analogous 10

to those d i s c u s s e d in s e ction 3 can be o b tained by p r o c e e d ­

ing in the same m a n n e r as above. Letting v-^(t) denote the

o ptimal g u a r a n t e e d e x p e c t e d p a y o f f to be received by player

1 from the r e maining T-t moves w h e n the game is in state k,

define v h (t~l) by v^ft-l) = Value (ak „.(t) + p\.V(t)). 1/(0)K K --------- Z-J

= (v^ (0 ) ,i>2 (0 ) » • • • »y^ ( ^ ) ) is> then, the lower vector value

of the finite stage game.