1. Nonlinear Estimation1 Formulas A formula is a (possibly) nonlinear mathematical relationship. A SERIES is a vector of numbers Formulas versus SET: –You can use SET to create the variable y as: SET y = 2 +.5*x**2 This creates a new series equal to two plus ½ of the square of x –A FORMULA can be used to express the mathematical relationship: FRML y =  +  x**  No variable is created—the three parameters can now be estimated

2. Nonlinear Estimation2 NNLS There are 4 statements required for nonlinear least squares: 1.indicate the coefficient to be estimated 2.Create the FRML to estimate 3.Initialize the coefficients 4.NLLS(FRML=name,OPTIONS) depvar start end residuals coeffs Standard Regression Options METHOD=[GAUSS]/SIMPLEX/GENETIC robusterrors iterations = Defined Variables Those of LINREG %FUNCVAL

3. Nonlinear Estimation3 Example 1 Example 1: Let x and y be series and suppose you want to estimate a, b, and r. nonlin a b r frml z = a + b*x**r compute a =.4, b = -.3, c = 2 nlls(frml=z) y

4. Nonlinear Estimation4 Since the disturbance term is additive, you cannot simply take the log of each side and estimate the equation using OLS. However, nonlinear least squares allows you to estimate  and  without transforming the variables. In RATS, you use the following structure to estimate a model using nonlinear least squares: NONLIN list of parameters to be estimated FRML formula name the equation to be estimated COMPUTE initial guesses for the parameters NNLS(FRML=formula name) dependent variable For the example at hand, you could use: nonlin alpha beta frml equation_1 y = alpha*x**beta com alpha = 1.0, beta = 0.5 nlls(frml = equation_1) y If the model had the form y t =  x t   t where {  t } is log-normal, it would be appropriate to estimate the regression in logs using LINREG.

5. Nonlinear Estimation5 NLLS(frml=formula name, other options) depvar start end residuals coeffs where: depvarDependent variable used on the FRML instruction. start endRange to estimate. residuals Series to store the residuals (Optional). coeffsSeries to store the estimated coefficients (Optional). The principal options are: METHOD =[GAUSS]/SIMPLEX/GENETIC. GAUSS requires a twice differential function. USE SIMPLEX if you have convergence problems. It is possible to use SIMPLEX or GENETIC to refine the initial guesses before using GAUSS. iterations = Maximum number of iterations to use. ROBUSTERRORS/As in LINREG, this option calculates a [NOROBUSTERRORS]consistent estimate of the covariance matrix.

6. Nonlinear Estimation6 Example 2 nonlin a b c frml z =.25*(a*a + b*b)*x{1} + a*sin1 + b*cos1 + c*dx{1} compute a =.4, b = -.3, c = 2 nlls(frml=z) dx Suppose that I want to estimate: dx t = 0.25*  x t-1 + a sin1 t + b cos1 t + c dx t-1 This can be done with LINREG: lin dx # x{1} sin1 cos1 dx{1} But suppose that I want to constrain  to equal a 2 + b 2

7. Nonlinear Estimation7 Controlling the Estimation NLPAR: You will not need to use NLPAR unless you experience convergence problems or want to obtain more precise numerical answers. Numerical optimization algorithms use iteration routines that cannot guarantee precise solutions for β. NLPAR allows you to select the various criteria RATS uses to determine when (and if) the solution converges. There are two principal options; the syntax is: nlpar(options) CRITERION = In the default mode, CRITERION=COEFFICIENTS. Here, convergence is determined using the change in the numerical value of the coefficients between iterations. Setting CRITERION=VALUE means that convergence is determined using the change in the value of the function being maximized. CVCRIT = Converge is assumed to occur if the change in the COEFFICIENTS or VALUE is less than the number specified. The default is 0.00001.

8. Nonlinear Estimation8 Examples of NLPAR 1. nlpar(cvcrit=0.0001) Setting CVCRIT=0.0001 means that RATS will continue to search for the values of the coefficients that maximize f( ) until the change in the coefficients between iterations is not more than 0.0001. 2. nlpar(criterion=value,cvcrit=0.0000001) Setting CVCRIT=0.0000001 and CRITERION=VALUE means that RATS will continue to search for the values of the coefficients that maximize f( ) until the change in the value of f( ) between iterations is less than 0.0000001.

9. Nonlinear Estimation9 Maximize Suppose your data set contains T observations of the variables y t and x t and you have used the FRML instruction to define the function: L = f(y t, x t ; β) where: x t and β can be vectors (and x t can represent a lagged value of y t ). MAXIMIZE is able to find the value(s) of β that solve: The syntax and principal options of MAXIMIZE are: maximize(options) frml start end funcval where:frml A previously defined formula start endThe range of the series to use in the estimation funcval(Optional) The series for the computed values of f(y t, x t ; β)

10. Nonlinear Estimation10 Options for MAXIMIZE The key options for our purposes are: METHOD = RATS is able to use any one of three different algorithms to find the maximum: BFGS, BHHH, or SIMPLEX. The technical details of each maximization algorithm are provided in the RATS manual. Use either the default BFGS method or the BHHH method for twice-differentiable functions and SIMPLEX in other cases. If you have convergence problems with BFGS, try BHHH. Note that SIMPLEX is extremely slow and often requires many iterations. ITERATIONS=The upper limit of the number of iterations used in the maximization. RECURSIVE Use this option if the formula must be solved recursively. It is necessary to use RECURSIVE if the value of the value of f( ) depends on the value of a variable at t-i. Note: You can use TEST and RESTRICT with the BFGS and BHHH options. Coefficients are numbered by their position in the NONLIN statement.

11. Nonlinear Estimation11 Maximum Likelihood Under the usual normality assumption, the log likelihood of observation t is: With T independent observations: We want to select  and   so as to maximize L

12. ML II Nonlinear Estimation12 Note that this is the same as maximizing: We want to select  and   so as to maximize L Which we would have obtained by writing the likelihood for obs. t as

13. Nonlinear Estimation13 Maximum Likelihood in RATS As in NLLS, to use MAXIMIZE, you must first: 1. List the name(s) of the parameters over which RATS is to perform the maximization. This is done using the NONLIN command. 2. Define the likelihood function f( ) using a FRML statement. 3. Set the initial values of the parameters using the COMPUTE command. Consider the following RATS statements used to estimate the linear regression y = a + bx: NONLIN b var FRML L = -log(var) - (y - b*x)**2/var COMPUTE b = initial guess, var = initial guess MAXIMIZE L start end OR, you can use to formulas frml e = y ‑ b*x frml L = ‑ log(var) - e**2/var

14. Nonlinear Estimation14 Subformulas y t =  t +  1  t-1 Since the {  t } sequence is unobserved, it is not possible to use LINREG or NLLS to estimate the process. To estimate  1 using maximum likelihood techniques, it is necessary to construct a formula of the form  t = y t -  1  t-1. However, the following is an illegal statement because e t is defined in terms of its own lagged value (a nonresolvable recursive expression): frml e = y – b1*e{1} The way to circumvent this problem is to create a “placeholder” series using the SET instruction. Then, define the desired formula in terms of the placeholder series. Finally, use a SUBFORMULA to equate the placeholder and the desired series. For example, a simple way to create the formula for the MA(1) process is: set temp = 0.0 nonlin b1 var frml e = y - b*temp{1} frml L = (temp = e), -log(var) - e**2/var)

15. Nonlinear Estimation15 Subformulas (cont.) drs t =  1 drs t-1 +  7 drs t-7 +  t +  1  t-1 +  2  t-2 Now, the NONLIN instruction contains the coefficient b2. The first FRML instruction uses temp{1} and temp{2} as placeholders for e t-1 and e t-2. The second FRML instruction creates the desired log likelihood and the COMPUTE instruction provides the initial guesses. Notice that the start date can remain at 9 since no usable observations are lost from the MA terms. set temp = 0. nonlin a1 a7 b1 b2 var frml e = drs - a1*drs{1} - a7*drs{7} - b1*temp{1} - b2*temp{2} frml L = (temp=e), -log(var) - (e)**2/var com a1 = 0.4, a7 = -.3, b1 =.5, b2 = 0.3, var = 1. max L 9 *

16. Nonlinear Estimation16 The GARCH Likelihood Function

17. Nonlinear Estimation17 ARMA(1,1)-IGARCH(1,1) set temp = 0. nonlin a0 a1 a2 a3 b0 b1 c1 b0.ge.0. b1.ge.0. c1.ge.0. frml e = spread - a0 - a1*spread{1} - a2*spread{2} - a3*spread{3} frml h = b0 + b1*e{1}**2 + c1*temp{1} frml L = (temp = h), -log(temp) - (e)**2/temp After initializing the parameters with the LINREG and COMPUTE instructions, the first MAXIMIZE instruction refines the initial guesses with the SIMPLEX method. The second obtains the final estimates using to BFGS method: lin(noprint) spread ; # constant spread{1 to 3} com a0 = %beta(1), a1 = %beta(2), a2 = %beta(3), a3 = %beta(4), b0 = %seesq, $ b1 = 0.2, c1 = 0.5 max(method=simplex,iters=5) L 7 * max(iters=200) L 7 *