Lecture 2: Introduction and Empirical framework for Markov ...shcherbakov.vwl.uni-mannheim.de/.../L2_EP_framework.pdfEP framework: setup Incumbent firms Firm i is characterized by

Lecture 2: Introduction and Empiricalframework for Markov perfect industry

dynamics

April 15, 2015

1 / 34

Introduction:

• Importance of strategic interactions: games vs single-agent problems

◦ In single-agent problems you search for a solution to a dynamicprogramming problem, i.e., value function is a fixed point of theBellman equation.

◦ In multi-agent models an equilibrium is a fixed point of a system ofbest response operators and each player’s best response is itself asolution to a dynamic programming problem.

• Several problems are specific to a game-theoretic environment◦ Existence of an equilibrium.◦ Multiplicity and selection of equilibria.◦ Computational burden.

2 / 34

Introduction:

• We seek for a framework that is consistent with several empiricalfacts about imperfectly competitive industries

◦ Simultaneous exit and entry.

◦ Heterogeneity among firms within a market.

◦ Firm- and market-level uncertainty.

◦ Correlations in the outcomes (profits) among firms in the sameindustry.

• We begin our discussion with a widely used framework: Ericson andPakes (1995).

• EP framework is a dynamic stochastic game with a discrete statespace.

3 / 34

Aside: why this is useful

• There are several methods that allow us to recover underlyingprimitives of dynamic games without ever solving for an equilibrium,e.g.,

◦ Bajari, Benkard, and Levin (2007),◦ Pakes, Ostrovsky, and Berry (2007),◦ Aguirregabiria and Mira (2007),◦ Pesendorfer and Schmidt-Dengler (2008), etc.

• Why do we bother to go through the hassle of computing one?◦ Academic curiosity◦ Counterfactual simulations

• Even if one never uses the model to estimate parameters of interestusing brute force approach (e.g., nested fixed point), computing theMarkov perfect equilibrium is necessary to evaluate various policies.

4 / 34

EP framework: setup

• A modified version of the Ericson-Pakes framework based onDoraszelski and Pakes (2007)

• Forward-looking firms, discrete time, infinite horizon.

• Firms maximize expected discounted sum of profit flows.

• At the beginning of each period, two sets of firms simultaneouslymake decisions:

1. Incumbent firms decide whether to stay in or not. Conditional onstaying - how much to invest.

2. Potential entrants decide whether to enter the market, and if theydo, how much to invest.

• Realizations of these decisions take place at the end of the timeperiod.

5 / 34

EP framework: setup

• Product market competition takes place◦ Outcomes in the product market do not affect the dynamics of the

industry

This is called “static-dynamic breakdown”. Does this always hold?

◦ Treat per-period profit function as a primitive of the dynamic game

• Per-period profits realize (depending on the state and actions)

• Timing:◦ Beginning of t: (1) investment, (2) entry, and (3) exit decisions are

made simultaneously

◦ End of t: realizations of the decisions made and product marketcompetition takes place (note: prior to exiting an incumbent collectscurrent period profit).

6 / 34

EP framework: setup

• Incumbent firms◦ Firm i is characterized by its state ωi ∈ Ω = {1, 2, 3, . . . , ω̄}

◦ Investment xi ≥ 0 may increase its state: ωi in the next period isstochastically increasing in xi .

At the same time, ωi may decrease by an exogenous negative shock.

◦ At the beginning of each period, incumbent firm i draws a randomand privately known scrap value φi from F .

We assume that φi is iid across firms and time. Why is this helpful?

◦ If i decides to exit, it competes in the product market in the lastperiod and then exits.

◦ We use ri to denote the probability that the incumbent firm i staysin the industry.

7 / 34

EP framework: setup

• Potential entrants◦ Finite number E of potential entrants in each time period.

◦ Potential entrants are short-lived, i.e. no strategic delays in entrydecisions.

◦ At the beginning of each period, each potential entrant draws arandom and privately known entry cost φei from F

e .

We assume φei are iid across firms and time.

◦ Upon entry, potential entrant i incurs φei and chooses its initialinvestment xei ≥ 0.

◦ It earns profits from the next period.

◦ We use r ei to denote probability that potential entrant i enters.

8 / 34

EP framework: setup

• States◦ A list of states of the incumbent firms ω = (ω1, . . . , ωn)

◦ A vector of scrap values φ = (φ1, . . . , φn), and

◦ A vector of entry costs φe = (φe1, . . . , φen)

completely characterize the industry

• To make computations easier, integrate out φ and φe : now ωcompletely characterizes the industry.

• Industry structure is given by

S = {(ω1, · · · , ωn) : ωi ∈ Ω, n ≤ n̄}

9 / 34

EP framework: Payoffs and strategies

• Payoffs and strategies are symmetric and anonymous◦ Symmetry: fi (ωi , ω−i ) = fj(ωi , ω−i )

◦ Anonymity (exchangeable): f (ωi , ω−i ) = f (ωi , ωπ(−i))

◦ Asymmetries between firms are then captured by the state variablesonly.

• This allows us to focus on a proper subset So of S :

So =

{(ωi , s) : ωi ∈ Ω, s = (s1, . . . , sω̄), sω̄ ∈ Z+,

∑ω∈Ω

sω ≤ n̄

}

where Z+ is the set of non-negative integers.

• Let π(ωi , ω−i ) denote the per-period profit in the product marketwhen the industry’s state is ω.

10 / 34

EP framework: Transitions

• Transition probability of firm i ’s state depends only on its owninvestment and current state (not on the investments or states of itscompetitors).

• Let η denote industry-wide shock, e.g. change in demand, inputcosts, technology, etc.

We need this to allow for positive correlation in profits of differentfirms in the same industry

• A family of probability distributions for ω′i given current state ωi ,firms’ choices, xi , and industry shocks η:

Pω′ ≡{p(·|ωi , xi , η) : ωi ∈ Ω, xi ∈ R+, η ∈ Υ

},

where we assume Pω′ is stochastically increasing in ωi and xi .

11 / 34


• Example of the evolution of ω

ω′i = ωi + νi − η,

where νi represents an outcome of the firm’s investment.

• Example of the investment outcome distribution

Pr(ν = 1|xi ) =αxi

1 + αxi, ν ∈ {0, 1}

12 / 34


13 / 34


• Consider a game of capacity accumulation with n = 2, ω̄ ≥ 3 states,and no entry/exit.

• In each period, firms invest and the likelihood of positive outcome isgiven by pi = Pr(ν = 1|xi ).

• An independent across firms depreciation shock hits a firm withprobability δ and if it does, it decreases the firm’s capacity by one.

• Let δ̄ = 1− δ then transition probabilities are given in the followingtable.

14 / 34


• Example from Doraszelski and Satterthwaite (2010)

ω′2 = ω2 + 1 ω′2 = ω2 ω

′2 = ω2 − 1

ω′1 = ω1 + 1 δ̄p1δ̄p2δ̄p1×[

δp2 + δ̄ (1− p2)] δ̄p1δ(1− p2)

ω′1 = ω1

δp1δ̄p2+δ̄ (1− p1)×δ̄p2

[δp1 + δ̄ (1− p1)

]×[

δp2 + δ̄ (1− p2)] [δp1 + δ̄ (1− p1)]

×δ(1− p2)

ω′1 = ω1 − 1 δ (1− p1) δ̄p2δ (1− p1)×[

δp2 + δ̄ (1− p2)] δ (1− p1) δ(1− p2)

15 / 34

EP framework: Incumbent’s problem

• Let V (ωi , ω−i , φ) denote the expected NPV of all future cash flowsof firm i , when the industry is characterized by ω and the firm hasdrawn φ. The Bellman equation is

V (ωi , ω−i , φ) =

π(ωi , ω−i ) + max

{φ,max

xi

(−xi + βE

[V (ω′i , ω

′−i , φ

′)|ωi , ω−i , xi])}

• Let q(ω′−i |ωi , ω−i , η) denote perceived probability of the next periodvalue of its competitors’ state ω′−i . Then,

E[V (ω′i , ω

′−i , φ

′)|ωi , ω−i , φ]

=∑ν

W (ν|ωi , ω−i )p(ν|xi ),

W (ν|ωi , ω−i ) ≡∑ω′−i ,η

∫φ′V (ωi + ν − η, ω′−i , φ′)dF (φ′)q(ω′−i |ωi , ω−i )p(η)

16 / 34


• Example for n = 2, E = 1, ω = (1, 2, 0) and η = 0.

For given policies r(ωi , ω−i ), re(ω), xi (ωi , ω−i ), i = 1, 2, and x

e(ω),q(ω′−i |ωi , ω−i , η) is calculated as

q(ω′−1 = (2, 1)|ω, η) = r(ω2, ω−2)(1− Pr(ν = 1|x2(ω2, ω−2)))×r e(ω) Pr(ν = 1|xe(ω))

17 / 34


• For given {W (·)}, the optimal investment behavior is characterizedby

xi

(β∑ν

W (ν|ωi , ω−i )∂p(ν|xi )∂xi

− 1

)= 0 ∧ xi ≥ 0

◦ Firm i selects an investment level equalizing current marginal costswith the marginal change in the expected present value of the statesthat might be realized next period.

• Let x(ωi , ω−i ) be an optimal investment policy and χ(ωi , ω−i , φ) bean indicator function which equals to one if the firm continues andzero otherwise.

18 / 34


• Then

χ(ωi , ω−i , φ) = arg maxχ∈{0,1}

(1− χ)φ

+ χ

(β∑ν

W (ν|ωi , ω−i )p(ν|x(ωi , ω−i ))− x(ωi , ω−i )

)

and

r(ωi , ω−i ) = F

(β∑ν


)

defines probability of staying in the industry.

19 / 34


• Using this, integrate φ to get

V (ωi , ω−i ) =π(ωi , ω−i ) + (1− r(ωi , ω−i ))E [φ|χ(ωi , ω−i , φ) = 0]

+ r(ωi , ω−i )

(β∑ν


)

• where

E [φ|χ(ωi , ω−i , φ) = 0] = E[φ|φ > F−1(r(ωi , ω−i ))

]=

1

1− r(ωi , ω−i )

∫φ>F−1(r(ωi ,ω−i ))

φdF .

20 / 34

EP framework: Entrant’s problem

• Value function for a potential entrant i is given by

V e(ω, φe) = max

{0,max

xei

[−φe − xei + β

∑ν

W e(ν|ω)p(ν|xei )

]}

where W e(·) is defined similar to W (·).

• Optimal investment policy is characterized by

xei

(β∑ν

W e(ν|ω)∂p(ν|xei )

∂xei− 1

)= 0 ∧ xei ≥ 0

• Let xe(ω) be i ’s optimal investment policy and χe(ω, φ) be theindicator function which equals to one if the potential entrant entersand zero otherwise.

21 / 34


• Then,

χe(ω, φe) = arg maxχ∈{0,1}

χ

(−φe − xe(ω) + β

∑ν

W e(ν|ω)p(ν|xe(ω))

)

and

r e(ω) = F e

(−xe(ω) + β

∑ν


)defines entry probability.

22 / 34


• Using this, integrate φ to get

V e(ω) = r e(ω)

[−E [φe |χe(ω, φe) = 1]− xe(ω) + β

∑ν


],

where

E [φe |χe(ω, φe) = 1] = E[φe |φe < F e−1(r e(ω))

]=

1

r e(ω)

∫φe

EP framework: Equilibrium

• We consider a Markov perfect equilibria (MPE)

• At each ω ∈ So each incumbent and each potential entrant◦ chooses optimal policies given its beliefs about future industry

structures,◦ these beliefs are consistent with the behavior of each agent’s

competitors

• More formally: MPE consists of {V (·),V e(·), χ(·), χe(·), x(·), xe(·)}at each ω ∈ So , such that◦ given policies {χ(·), χe(·), x(·), xe(·)}, V (·) solves the Bellman

equation for incumbent firms, and V e(·) solves the Bellman equationfor entrants ∀ω ∈ So .

◦ given value functions V and V e , the policies for both incumbentsand potential entrants satisfy the optimality conditions.

24 / 34

EP framework: Equilibrium existence

A1. Boundedness of primitives:

(i) finite state space, ω̄


• Proposition 2. Under Assumptions 1,2, and 3, an equilibrium existsin cutoff entry/exit and pure investment strategies. (Doraszelski andSatterthwaite (2010))

◦ proof relies on a continuous mapping from policies into themselves(Brouwer’s fixed-point theorem);

◦ adding random scrap values and entry costs allows us to treat thecontinuous exit and entry probabilities as policies;

◦ given strategies of competitors, a firm solves a single-agent decisionproblem, can use dynamic programming techniques (contractionmapping) to show that the firm’s best reply is well-defined.

26 / 34


• Example of non-existence with deterministic scrap values fromDoraszelski and Satterthwaite (2010)

◦ Consider a war of attrition with a common and constant exit value φ.

◦ No entry, state ω = (ω1, ω2), where ωi = 1 denotes active andωi = 2 inactive state.

◦ Profits:

π(ωi , ω−i ) =

{π(1, 1) duopoly

π(1, 2) monopoly

◦ Let φ be s.t.βπ(1, 1)

1− β < φ <βπ(1, 2)

1− β .

27 / 34


◦ Given firm 2’s decision χ(1, 1) ∈ {0, 1}, firm 1’s values are

V (1, 2) = supχ̃(1,2)∈{0,1}

π(1, 2) + (1− χ̃(1, 2))φ+ χ̃(1, 2)βV (1, 2)

V (1, 1) = supχ̃(1,1)∈{0,1}

π(1, 1) + (1− χ̃(1, 1))φ

+ χ̃(1, 1)β {χ(1, 1)V (1, 1) + (1− χ(1, 1))V (1, 2)}

◦ Firm 1’s optimal exit decision χ̃(1, 2) and χ̃(1, 1) satisfy

χ̃(1, 2) = 1{φ ≤ βV (1, 2)} (1)χ̃(1, 1) = 1{φ ≤ β[χ(1, 1)V (1, 1) + (1− χ(1, 1))V (1, 2)]} (2)

◦ Consider a symmetric equilibrium in pure exit strategies.

28 / 34


◦ Suppose χ(1, 2) = 0. Then V (1, 2) = π(1, 2) + φ and (1) implies

φ ≥ β(π(1, 2) + φ)⇒ φ ≥ βπ(1, 2)1− β ,

contradiction.

◦ Suppose χ(1, 1) = 1. Then V (1, 1) = π(1,1)1−β and (2) implies

φ ≤ βπ(1, 1)1− β ,

contradiction.

◦ Finally, χ(1, 2) = 1 and χ(1, 1) = 0. Then V (1, 2) = π(1,2)1−β and (2)

implies

φ ≥ βV (1, 2)⇒ φ ≥ βπ(1, 2)1− β

contradiction.

29 / 34


• To ensure existence consider random scrap values and entry costs• Setup is the same except that scrap values now are given by φ+ �θi ,

where φ is a constant and θi ∼ F (·) with E [θ] = 0.• Private information is θ and � is a constant that measures the

importance of incomplete information.

• Bellman equation

V (1, 2) = supr̃(1,2)∈[0,1]

π(1, 2) + (1− r̃(1, 2))φ

+ �

∫θ>F−1(r̃(1,2))

θdF (θ) + r̃(1, 2)βV (1, 2)

V (1, 1) = supr̃(1,1)∈[0,1]

π(1, 1) + (1− r̃(1, 1))φ

+ r̃(1, 1)β[r(1, 1)V (1, 1) + (1− r(1, 1))V (1, 2)]

30 / 34


• Optimal exit decisions of firm 1 are given by

r̃(1, 2) = F

(βV (1, 2)− φ

�

)r̃(1, 1) = F

(β[r(1, 1)V (1, 1) + (1− r(1, 1))V (1, 2)]− φ

�

)• In a symmetric equilibrium, r̃(ω1, ω2) = r(ω2, ω1), yielding a system

of four equations with four unknowns: V (1, 2), V (1, 1), r(1, 2), andr(1, 1)

• Note that equilibrium with random scrap values converges to theequilibrium in mixed strategies as � approaches zero.

31 / 34

EP framework: Equilibrium uniqueness

• In general, multiple equilibria cannot be excluded.

• Three reasons (see examples in the online Appendix to Doraszelskiand Satterthwaite (2010))

◦ investment decisions,

◦ entry/exit decisions,

◦ product market competition.

• How to deal with them?(1) Assume away.

(2) Change endogenous variables so there exists a unique equilibrium interms of the new endogenous variable in question.

(3) Use set-identification.

32 / 34

EP framework: Important properties

• Equilibrium is characterized by substantial inflows and outflows offirms

• Multiple rank reversals are possible during the life of any collectionof firms

• All firms die in finite time almost surely, but new firms continuallyenter

• Ericson and Pakes (1995) prove that the state space S , contains aunique, positive recurrent communicating class R ⊂ S ,◦ Define µn(v) as the probability that the structure of the industry

with initial distribution v is in state s after n periods, then thereexists a unique, invariant probability measure, µ∗, on S s.t.

∀s ∈ S , µn(s) −→n→∞

µ∗

µ∗s = 0 ∀s ∈ S\R

and the distribution of a stationary ergodic Markov process, Pµ∗ ,with transition Q s.t. µ∗Q = µ∗.

33 / 34

EP framework: Important properties

• Implications of ergodicity (in our context we can say that timeaverages approximate state averages)

◦ All industry structures in the recurrent class R ⊂ S are realizedinfinitely often. R is often much smaller than S . Note that theindustry evolves in a non-degenerate, but increasingly regular way,i.e. there is never a “limit” structure of the industry.

◦ After some time, a certain stochastic regularity will appear in theevolution of the industry.

◦ Effects of any initial situation systematically fade. Two possiblehistories for the industry with different initial conditions, once theyintersect in any state, have identical distributions over future samplepaths conditional on that intersection.

34 / 34

Aguirregabiria, Victor and Pedro Mira (2007), “Sequential estimation of dynamic discrete games.” Econometrica, 75, 1–53.

Bajari, Patric, Lanier Benkard, and Jonathan Levin (2007), “Estimating dynamic models of imperfect competition.” Econometrica, 75,1331–1370.

Doraszelski, Ulrich and Ariel Pakes (2007), “A framework for applied dynamic analysis in IO.” In Handbook of Industrial Organization(M. Armstrong and R. Porter, eds.), volume 3, chapter 30, 1887–1966, Elsevier B.V.

Doraszelski, Ulrich and Mark Satterthwaite (2010), “Computable Markov-perfect industry dynamics.” RAND Journal of Economics, 41,215–243.

Ericson, Richard and Ariel Pakes (1995), “Markov-perfect industry dynamics: a framework for empirical work.” The Review of EconomicStudies, 62, 53–82.

Pakes, Ariel, Michael Ostrovsky, and Steven Berry (2007), “Simple estimators for the parameters of discrete dynamic games (withentry/exit examples).” RAND Journal of Economics, 38, 373–399.

Pesendorfer, Martin and Philipp Schmidt-Dengler (2008), “Asymptotic least squares estimators for dynamic games.” The Review ofEconomic Studies, 75, 901–928.

34 / 34

IntroductionAsidewhy this is useful

EP frameworksetupPayoffs and strategiesTransitionsIncumbent's problemEntrant's problemEquilibriumEquilibrium existenceEquilibrium uniquenessImportant properties

Documents

Lecture 2: Introduction and Empirical framework for Markov ...shcherbakov.vwl.uni-mannheim.de/.../L2_EP_framework.pdfEP framework: setup Incumbent firms Firm i is characterized by