Nonlinear Estimation1 Formulas A formula is a (possibly) nonlinear mathematical relationship. A SERIES is a vector of numbers Formulas versus SET: –You

Nonlinear Estimation 1

Formulas

• A formula is a (possibly) nonlinear mathematical relationship. A SERIES is a vector of numbers

• Formulas versus SET:– You can use SET to create the variable y as:

• SET y = 2 + .5*x**2

• This creates a new series equal to two plus ½ of the square of x

– A FORMULA can be used to express the mathematical relationship:

• FRML y = + x**• No variable is created—the three parameters can now be estimated


NNLSThere are 4 statements required for nonlinear least squares:

1. indicate the coefficient to be estimated2. Create the FRML to estimate3. Initialize the coefficients4. NLLS(FRML=name ,OPTIONS) depvar start end residuals coeffs

Standard Regression OptionsMETHOD=[GAUSS]/SIMPLEX/GENETICrobusterrorsiterations =

Defined VariablesThose of LINREG%FUNCVAL


Example 1

Example 1:Let x and y be series and suppose you want to estimate a, b, and r.

nonlin a b rfrml z = a + b*x**rcompute a = .4, b = -.3, c = 2nlls(frml=z) y


Since the disturbance term is additive, you cannot simply take the log of each side and estimate the equation using OLS. However, nonlinear least squares allows you to estimate and without transforming the variables. In RATS, you use the following structure to estimate a model using nonlinear least squares: NONLIN list of parameters to be estimatedFRML formula name the equation to be estimatedCOMPUTE initial guesses for the parametersNNLS(FRML=formula name) dependent variable For the example at hand, you could use: nonlin alpha betafrml equation_1 y = alpha*x**betacom alpha = 1.0 , beta = 0.5nlls(frml = equation_1) y If the model had the form yt = xt

t where {t} is log-normal, it would be

appropriate to estimate the regression in logs using LINREG.


NLLS(frml=formula name, other options) depvar start end residuals coeffs

where: depvar Dependent variable used on the FRML instruction.start end Range to estimate.residuals Series to store the residuals (Optional).coeffs Series to store the estimated coefficients (Optional).

The principal options are: METHOD = [GAUSS]/SIMPLEX/GENETIC. GAUSS requires a twice differential function. USE SIMPLEX if you have convergence problems. It is possible to use SIMPLEX or GENETIC to refine the initial guesses before using GAUSS.iterations = Maximum number of iterations to use.ROBUSTERRORS/ As in LINREG, this option calculates a [NOROBUSTERRORS]consistent estimate of the covariance matrix.


Example 2

nonlin a b cfrml z = .25*(a*a + b*b)*x{1} + a*sin1 + b*cos1 + c*dx{1} compute a = .4, b = -.3, c = 2nlls(frml=z) dx

Suppose that I want to estimate:dxt = 0.25* xt-1+ a sin1t + b cos1t + c dxt-1

This can be done with LINREG: lin dx# x{1} sin1 cos1 dx{1}

But suppose that I want to constrain to equal a2 + b2


Controlling the EstimationNLPAR: You will not need to use NLPAR unless you experience convergence problems or want to obtain more precise numerical answers. Numerical optimization algorithms use iteration routines that cannot guarantee precise solutions for β. NLPAR allows you to select the various criteria RATS uses to determine when (and if) the solution converges. There are two principal options; the syntax is: nlpar(options)CRITERION = In the default mode, CRITERION=COEFFICIENTS. Here, convergence is determined using the change in the numerical value of the coefficients between iterations. Setting CRITERION=VALUE means that convergence is determined using the change in the value of the function being maximized.CVCRIT = Converge is assumed to occur if the change in the COEFFICIENTS or VALUE is less than the number specified. The default is 0.00001.


Examples of NLPAR

1. nlpar(cvcrit=0.0001) Setting CVCRIT=0.0001 means that RATS will continue to search for the values of the coefficients that maximize f( ) until the change in the coefficients between iterations is not more than 0.0001. 2. nlpar(criterion=value,cvcrit=0.0000001) Setting CVCRIT=0.0000001 and CRITERION=VALUE means that RATS will continue to search for the values of the coefficients that maximize f( ) until the change in the value of f( ) between iterations is less than 0.0000001.


MaximizeSuppose your data set contains T observations of the variables yt and xt and you

have used the FRML instruction to define the function: L = f(yt , xt; β)

where: xt and β can be vectors (and xt can represent a lagged value of yt).

MAXIMIZE is able to find the value(s) of β that solve:The syntax and principal options of MAXIMIZE are: maximize(options) frml start end funcvalwhere: frml A previously defined formula

start end The range of the series to use in the estimationfuncval (Optional) The series for the computed values of f(yt, xt; β)


Options for MAXIMIZE

The key options for our purposes are:

METHOD = RATS is able to use any one of three different algorithms to find the maximum: BFGS, BHHH, or SIMPLEX. The technical details of each maximization algorithm are provided in the RATS manual. Use either the default BFGS method or the BHHH method for twice-differentiable functions and SIMPLEX in other cases. If you have convergence problems with BFGS, try BHHH. Note that SIMPLEX is extremely slow and often requires many iterations.

ITERATIONS= The upper limit of the number of iterations used in the maximization.

RECURSIVE Use this option if the formula must be solved recursively. It is necessary to use RECURSIVE if the value of the value of f( ) depends on the value of a variable at t-i. Note: You can use TEST and RESTRICT with the BFGS and BHHH options. Coefficients are numbered by their position in the NONLIN statement.


Maximum Likelihood

2 22

1(1/ 2) ln(2 ) (1/ 2) ln ( )

2tt y x

Under the usual normality assumption, the log likelihood ofobservation t is:

With T independent observations:

2 22

1log ln(2 ) ln ( )

2 2 2

T

ttt=1

T TL = y x

We want to select and so as to maximize L

ML II


2 22

1

1log ln ( )

T

ttt=

L = T y x

Note that this is the same as maximizing:

We want to select and so as to maximize L

2 22

1

1log ln(2 ) ln ( )

2 2 2

T

ttt=

T TL = y x

2 22

1ln ( )tt y x

Which we would have obtained by writing the likelihood for obs. t as


Maximum Likelihood in RATSAs in NLLS, to use MAXIMIZE, you must first:1. List the name(s) of the parameters over which RATS is to perform the maximization. This is done using the NONLIN command. 2. Define the likelihood function f( ) using a FRML statement. 3. Set the initial values of the parameters using the COMPUTE command. Consider the following RATS statements used to estimate the linear regression y = a + bx: NONLIN b varFRML L = -log(var) - (y - b*x)**2/varCOMPUTE b = initial guess, var = initial guessMAXIMIZE L start end

OR, you can use to formulasfrml e = y ‑ b*xfrml L = ‑log(var) - e**2/var


Subformulas

yt = t + 1t-1 Since the {t} sequence is unobserved, it is not possible to use LINREG or NLLS to estimate the

process. To estimate 1 using maximum likelihood techniques, it is necessary to construct a

formula of the form t = yt - 1t-1. However, the following is an illegal statement because et is

defined in terms of its own lagged value (a nonresolvable recursive expression): frml e = y – b1*e{1}

The way to circumvent this problem is to create a “placeholder” series using the SET instruction. Then, define the desired formula in terms of the placeholder series. Finally, use a SUBFORMULA to equate the placeholder and the desired series. For example, a simple way to create the formula for the MA(1) process is: set temp = 0.0nonlin b1 varfrml e = y - b*temp{1}frml L = (temp = e), -log(var) - e**2/var)


Subformulas (cont.)

drst = 1drst-1 + 7drst-7 + t + 1t-1 + 2t-2

Now, the NONLIN instruction contains the coefficient b2. The first FRML instruction uses temp{1} and temp{2} as placeholders for et-1 and et-2. The second

FRML instruction creates the desired log likelihood and the COMPUTE instruction provides the initial guesses. Notice that the start date can remain at 9 since no usable observations are lost from the MA terms. set temp = 0.nonlin a1 a7 b1 b2 varfrml e = drs - a1*drs{1} - a7*drs{7} - b1*temp{1} - b2*temp{2}frml L = (temp=e), -log(var) - (e)**2/varcom a1 = 0.4, a7 = -.3, b1 = .5, b2 = 0.3, var = 1.max L 9 *


The GARCH Likelihood Function

222

1

1ln ln (2 ) ln ( )

2 2 2

T

tt=

T TL =

2

1

1exp

22

Tt

t tt

Lhh

2

1 1

ln ln (2 ) 0.5 ln 0.5 ( / )2

TT

t t tt t=

TL = h h


ARMA(1,1)-IGARCH(1,1)

set temp = 0.nonlin a0 a1 a2 a3 b0 b1 c1 b0.ge.0. b1.ge.0. c1.ge.0. frml e = spread - a0 - a1*spread{1} - a2*spread{2} - a3*spread{3}frml h = b0 + b1*e{1}**2 + c1*temp{1}frml L = (temp = h), -log(temp) - (e)**2/temp

After initializing the parameters with the LINREG and COMPUTE instructions, the first MAXIMIZE instruction refines the initial guesses with the SIMPLEX method. The second obtains the final estimates using to BFGS method: lin(noprint) spread ; # constant spread{1 to 3}com a0 = %beta(1), a1 = %beta(2) , a2 = %beta(3), a3 = %beta(4) , b0 = %seesq, $b1 = 0.2, c1 = 0.5max(method=simplex,iters=5) L 7 *max(iters=200) L 7 *

Documents

Nonlinear Estimation1 Formulas A formula is a (possibly) nonlinear mathematical relationship. A SERIES is a vector of numbers Formulas versus SET: –You