Upload
mohamed-jasem
View
226
Download
0
Embed Size (px)
Citation preview
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
1/26
IntroductionJaynes Dice Problem
GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix
Sign and Cross-Parameter RestrictionsConclusion
Inequality restricted maximum entropy estimationin Stata
Randall Campbelly, R. Carter Hillz
Mississippi State Universityy, Louisiana State Universityz
Stata Conference New Orleans July 18, 2013
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
1
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
2/26
IntroductionJaynes Dice Problem
GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix
Sign and Cross-Parameter RestrictionsConclusion
Generalized Maximum Entropy Estimation
GME estimator developed by Golan, Judge, and Miller (1996)
Campbell and Hill (2006) impose inequality restrictions onGME estimator in a linear regression model
We develop new STATA commands to obtain GME parameterestimates, with or without inequality restrictions
Our commands utilizes the optimize() function in MATA
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
2
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
3/26
IntroductionJaynes Dice Problem
GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix
Sign and Cross-Parameter RestrictionsConclusion
Maximum Entropy Problem
Entropy is the amount of uncertainty represented by a discreteprobability distribution
Entropy is measured as H (p ) = p k ln(p k )
Jaynes dice problem: Estimate unknown probabilities of rolling each value on a die
Maximize H (p ) subject to: k p k = y and p k = 1,
where k is the number of sides on die and y is the mean fromprior rolls
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
3
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
4/26
IntroductionJaynes Dice Problem
GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix
Sign and Cross-Parameter RestrictionsConclusion
Maximum Entropy Solution
L = p k ln(p k ) + (y k p k ) + (1 p k )
dL/ dp k = 1 ln(p k ) k = 0
Which implies bp k = exp( b k ) exp( b k )Substitute into Lagrangian and minimize (Golan, Judge, and
Miller (1996)):
L( ) = bp k ( b ) ln( bp k ( b )) + b (y k bp k ( b ))Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
4
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
5/26
I d i
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
6/26
IntroductionJaynes Dice Problem
GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix
Sign and Cross-Parameter RestrictionsConclusion
STATA MEDICE Command
The part that evaluates the function:void mydice_eval(todo, lam, x, k, y, L, g, H){a = J(k,1,.)b = J(k,1,.)for (i=1; i< =k; i++)
{a[i] = exp(-x[i]*lam)}L = lam*y + ln(sum(a))
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
6
I t d ti
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
7/26
IntroductionJaynes Dice Problem
GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix
Sign and Cross-Parameter RestrictionsConclusion
Example
. scalar k = 6
. scalar y = 4.5
. medice k y
entropy function = 1.6135811
r6 .34749406
r5 .23977444
r4 .1654468r3 .11415998
r2 .07877155
r1 .05435317
c1
r(phat)[6,1]
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
7
Introduction
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
8/26
IntroductionJaynes Dice Problem
GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix
Sign and Cross-Parameter RestrictionsConclusion
Model Specication for GME Estimation
Model: y = X + e
Set up such that unknown parameters in form of probabilities(GJM, 1996):
= Zp =2664
z 01 0 ... 00 z
02 ... 0
. . . .
0 0 ... z 0k
3775
.
2664
p 1p 2
.
p k
3775
Do same thing with error term
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
8
Introduction
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
9/26
IntroductionJaynes Dice Problem
GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix
Sign and Cross-Parameter RestrictionsConclusion
Generalized Entropy Function
Reparameterized Model: y = XZp + Vw
Z and V are parameter and error support matrices,respectivelyMax H (p , w ) = p 0 ln(p ) w 0 ln(w ), subject toy = XZp + Vw (I K
Oi
0
M )p = i K
(I N Oi 0J )w = i N ,where M is # support points for each parameter; J is #support points for errors
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
9
Introduction
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
10/26
IntroductionJaynes Dice Problem
GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix
Sign and Cross-Parameter RestrictionsConclusion
GME Solution in Linear Regression Model
L = p 0 ln(p ) w 0 ln(w ) + 0(XZp + Vw y )+
0[i K (I K Oi 0M )p ] + 0[i N (I N Oi
0J )w ]
Solution:
bp km = exp(z km x
0k
b ) /
M
m = 1
exp(z km x 0k
b ), and
bw nj = exp(v nj b n ) /J
j = 1
exp(v nj b n )
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
10
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
11/26
Introduction
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
12/26
Jaynes Dice ProblemGME Estimation in Linear Regression Model
GME Command with User Supplied Parameter Support MatrixSign and Cross-Parameter Restrictions
Conclusion
Example using Generated Data
With the automated command, the user types gmereg y x1 x2x3 ...
As an example, we generate 7 standard normal randomvariables (x1-x7) and a standard normal error
y = 2x 1 + 1. 2x 2 + 0. 35x 3 + 0. 4x 4 0. 05x 5 + 0. 8x 6 3x 7 + e
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
12
Introduction
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
13/26
Jaynes Dice ProblemGME Estimation in Linear Regression Model
GME Command with User Supplied Parameter Support MatrixSign and Cross-Parameter Restrictions
Conclusion
Results
Results obtained using REG and GMEREG in STATA:true reg gmereg
x1 2.000 1.974 1.948x2 1.200 1.149 1.160x3 0.350 0.218 0.203x4 0.400 0.134 0.108
x5 -0.050 0.060 0.016x6 0.800 0.822 0.791x7 -3.000 -3.198 -3.194const 0.000 -0.125 -0.110
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
13
Introduction
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
14/26
Jaynes Dice ProblemGME Estimation in Linear Regression Model
GME Command with User Supplied Parameter Support MatrixSign and Cross-Parameter Restrictions
Conclusion
STATA GMEREGM Command
GMEREG automatically sets up the parameter support matrix
based on initial OLS estimates
A second version, GMEREGM, allows the user to specify theirown support matrix
This allows the user to specify wider or narrower bounds and,as we will show, to impose cross-parameter restrictions
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
14
Introductionbl
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
15/26
Jaynes Dice ProblemGME Estimation in Linear Regression Model
GME Command with User Supplied Parameter Support MatrixSign and Cross-Parameter Restrictions
Conclusion
Specifying the Parameter Support Matrix
Suppose we wish to impose wide, uninformative bounds; alloweach parameter to fall between -20 and 20
In our example, we have K = 8 parameters to estimate andhave M = 5 support points. We specify the K x M supportmatrix:. matrix zmat = (-20,-20,-20,-20,-20,-20,-20,-20 n
-10,-10,-10,-10,-10,-10,-10,-10 n0,0,0,0,0,0,0,0 n10,10,10,10,10,10,10,10 n20,20,20,20,20,20,20,20)
. gmeregm y x1 x2 x3 x4 x5 x6 x7
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
15
IntroductionJ Di P bl
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
16/26
Jaynes Dice ProblemGME Estimation in Linear Regression Model
GME Command with User Supplied Parameter Support MatrixSign and Cross-Parameter Restrictions
Conclusion
Alternative Specications
Suppose we wish a more informative prior:. matrix zmat = (-5,-5,-1,-1,-1,-1,-5,-5 n
-2.5,-2.5,-0.5,-0.5,-0.5,-0.5,-2.5,-2.5 n0,0,0,0,0,0,0,0 n2.5,2.5, 0.5,0.5,0.5,0.5,2.5,2.5 n5,5,1,1,1,1,5,5)
Or one that is not symmetric about zero:
. matrix zmat = (-10,-10,-1,-1,-1,-1,-15,-10 n0,0,0,0,-0.5,0,10,-5 n5,5,1,1,0,1,-5,0 n10,10, 2,2,0.5,2,0,5 n15,15,3,3,1,3,10,10)
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
16
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
17/26
IntroductionJaynes Dice Problem
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
18/26
Jaynes Dice ProblemGME Estimation in Linear Regression Model
GME Command with User Supplied Parameter Support MatrixSign and Cross-Parameter Restrictions
Conclusion
Sign Restrictions
Our estimates all took correct signs except the estimate for 5
Parameter sign restrictions are easily accomplished throughsupport matrix. For example,. matrix zmat = (0,0,0,0,0.4,0,-10,-10 n
2.5,1,0.25,0.25,-0.3,0.5,7.5,-5 n5,2,0.5,0.5,-0.2,1,-5,0 n7.5,3,0.75,0.75,-0.1,1.5,-2.5,5 n10,4,1,1,0,2,0,10)
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
18
IntroductionJaynes Dice Problem
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
19/26
Jaynes Dice ProblemGME Estimation in Linear Regression Model
GME Command with User Supplied Parameter Support MatrixSign and Cross-Parameter Restrictions
Conclusion
Cross-Parameter Restrictions
Note that while the true 4 > 3, this does not hold for ourestimates
Suppose we wish to restrict b 4 >
b 3Campbell and Hill (2006) impose such restrictions through thesupport matrix:
3 4 = Z
p 3p 4 =
z 03 0
z 03 z 04
p 3p 4
Specifying z 04 to take only non-negative values ensures that
b 4 >
b 3
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
19
IntroductionJaynes Dice Problem
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
20/26
Jaynes Dice ProblemGME Estimation in Linear Regression Model
GME Command with User Supplied Parameter Support MatrixSign and Cross-Parameter Restrictions
Conclusion
GME Solution with Cross-Parameter Restrictions
The problem becomes harder computationally; the GMEsolution is
bp km = exp(z km x
01 b + z km x
02 b + ... + z km x
0K b )M
m = 1
exp(z km x 01 b + z km x
02 b + ... + z km x
0K b )
When parameter support is block diagonal, cross-productterms drop out
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
20
IntroductionJaynes Dice Problem
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
21/26
Jay es ce ob eGME Estimation in Linear Regression Model
GME Command with User Supplied Parameter Support MatrixSign and Cross-Parameter Restrictions
Conclusion
Restricted GME Estimation
A third version, GMEREGMR, performs restricted estimation.However, must enter larger zmat (KM x K)Enter separate support column vector for each parameter (and
a vector of 0s). matrix z0 = (0 n0n0n0n0). matrix z1 = (0 n5n10n15n20). matrix z2 = z1
. matrix z3 = z1. matrix z4 = (0 n0.5n1n1.5n2)
. matrix z5 = (-20 n-15n-10n-5n0)
. matrix z6 = z1
. matrix z7 = z5
. matrix z8 = (-20 n-10n0n10n20)Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
21
IntroductionJaynes Dice Problem
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
22/26
yGME Estimation in Linear Regression Model
GME Command with User Supplied Parameter Support MatrixSign and Cross-Parameter Restrictions
Conclusion
STATA GMEREGMR Command
Now combine into zmat; matrix is block diagonal except forcolumn 4 which include z3 and z4. matrix zmat = (z1,z0,z0,z0,z0,z0,z0,z0 n
z0,z2,z0,z0,z0,z0,z0,z0 nz0,z0,z3,z3 ,z0,z0,z0,z0 nz0,z0,z0,z4 ,z0,z0,z0,z0 nz0,z0,z0,z0,z5,z0,z0,z0 nz0,z0,z0,z0,z0,z6,z0,z0 nz0,z0,z0,z0,z0,z0,z7,z0 nz0,z0,z0,z0,z0,z0,z0,z8)
Now enter command gmeregmr = y x1 x2 x3 x4 x5 x6 x7
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
22
IntroductionJaynes Dice Problem
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
23/26
GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix
Sign and Cross-Parameter RestrictionsConclusion
Final Example - Multiple Restrictions
Suppose we wish to impose 4 > 3 and 6 > 3 + 4 and 2 > 3 + 6The parameter support vectors we chose:
. matrix z0 = (0 n0n0n0n0)
. matrix z1 = (0 n5n10n15n20)
. matrix z2 = (0 n0.5n1n1.5n2)
. matrix z3 = z1
. matrix z4 = z2
. matrix z5 = (-20 n-15n-10n-5n0)
. matrix z6 = z2
. matrix z7 = z5
. matrix z8 = (-20 n-10n0n10n20)Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
23
IntroductionJaynes Dice Problem
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
24/26
GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix
Sign and Cross-Parameter RestrictionsConclusion
Zmat - Multiple Restrictions
The zmat to ensure restrictions hold. matrix zmat = (z1,z0,z0,z0,z0,z0,z0,z0 n
z0,z2 ,z0,z0,z0,z0,z0,z0 nz0,3*z3 ,z3,z3 ,z0,2*z3 ,z0,z0 nz0,z4 ,z0,z4 ,z0,z4 ,z0,z0 nz0,z0,z0,z0,z5,z0,z0,z0 nz0,z6 ,z0,z0,z0,z6 ,z0,z0 nz0,z0,z0,z0,z0,z0,z7,z0 nz0,z0,z0,z0,z0,z0,z0,z8)
Note: b 4 = b 3 + z 04p 4 > b 3 ; b 6 = b 3 + b 4 + z
06p 6 > b 3+ b 4 ; and b 2 = b 3 + b 6 + z
02p 2 > b 3+ b 6
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
24
IntroductionJaynes Dice Problem
GME E i i i Li R i M d l
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
25/26
GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix
Sign and Cross-Parameter RestrictionsConclusion
Results
Results based on dierent support matrices:true sign only single multiple
x1 2.000 1.961 2.009 2.067x2 1.200 1.217 1.250 1.324x3 0.350 0.363 0.165 0.006x4 0.400 0.347 0.385 0.224x5 -0.050 -0.150 -0.021 -0.041x6 0.800 0.849 0.952 0.860x7 -3.000 -3.220 -3.186 -3.194const 0.000 -0.125 -0.139 -0.163
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
25
IntroductionJaynes Dice Problem
GME E ti ti i Li R g i M d l
8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT
26/26
GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix
Sign and Cross-Parameter RestrictionsConclusion
Work to Do
Our commands provide GME estimates and can be used with
either no user input or a user-specied parameter support
We are working to output additional statistics such as priormean and standard errors
We are also developing STATA GME commands for binaryand multinomial choice models, censroed regression, ..
Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata
26