References for Computing Coefficients of the ARMA model.

We hope to come up with Python codes for estimating the coefficients of the ARMA AutoRegressive Moving Average time series model. Our sources are online documentation of existing softwares, forums and mailing lists.

1. Tom Doan shows a Rats code for Hannan-Rissanen algorithm: from Estima.com forum
   
   

   >
   * @HannanRissanen( options ) series start end
   *
   * Computes estimates for an ARMA model using the Hannan-Rissanen algorithm, which
   * runs a LS regression on lags of the dependent variable and residuals from a
   * long preliminary AR.
   *
   * Parameters:
   * series Series to analyze
   * start end Range to analyze [range of series]
   *
   * Options
   * AR=number of AR parameters [0]
   * MA=number of MA parameters [0]
   * DIFF=number of regular differencings[0]
   * SDIFFS=number of seasonal differencings[0]
   * M=number of lags in the long preliminary AR. By default, this uses the
   * Gomez proposal of max(log**2(T),2*max(p,q))
   * METHOD=[YULE]/BURG
   * Method used in computing the long AR
   * [CORRECTION]/NOCORRECTION
   * Chooses whether or not to do a one-step correction to the initial estimates.
   * [PRINT]/NOPRINT
   * DEFINE=equation to define
   *
   * References:
   * Hannan and Rissanen, "Recursive estimation of mixed autoregressive-moving average order",
   * Biometrika, 1991, vol 69, pp 81-94.
   *
   * Gomez and Maravall, "Automatic Modeling Methods for Univariate Series", in
   * Peña, Tiao and Tsay, eds., "A Course in Time Series Analysis", New York: Wiley, 2001.
   *
   procedure HannanRissanen series start end
   type series series
   type integer start end
   *
   option integer ar 0
   option integer ma 0
   option integer diffs 0
   option integer sdiffs 0
   option integer m
   option switch correction 1
   option choice method 1 yule burg
   option switch print 1
   option equation *define
   *
   local integer startl endl lags nobs skips span
   local series xc e v w
   local real value
   local equation armaeq
   local vector beta0
   *
   if .not.%defined(series) {
   disp "Syntax: @HannanRissanen(options) SERIES start end"
   return
   }
   
   compute span=2:1-1:1
   inquire(series=series) startl<>, do one step of a Gauss-Newton algorithm on the maximum sample
   * size that can be done by a LS regression. The definition of <> below (which
   * generates the derivatives with respect to the AR parameters) is more accurate than
   * what is described in (e.g.) Gomez and Maravall. This, in fact, gives the exact
   * derivatives of the <> series as generated below with respect to the AR parameters.
   *
   if correction {
   set e startl endl = %if(t
   

   
   The code still looks Greek to me at the moment.
   
   Mathematica Documentation on Time Series:
   
   The cached web page is at Google cached page.
   
   
   

2. Documentation from R of its arma() in the tseries function.
   The tseries package provides the arma() function. It uses the Hannan-Rissanen algorithm to provide initial estimates. R is an open source program and we can also view the code! Here is a short description:
   
   

   The following parametrization is used for the ARMA(p,q) model:
   
   y[t] = a[0] + a[1]y[t-1] + ... + a[p]y[t-p] + b[1]e[t-1] + ... + b[q]e[t-q] + e[t],
   
   where a[0] is set to zero if no intercept is included. By using the argument lag, it is possible to fit a parsimonious submodel by setting arbitrary a[i] and b[i] to zero.
   
   arma uses optim to minimize the conditional sum-of-squared errors. The gradient is computed, if it is needed, by a finite-difference approximation. Default initialization is done by fitting a pure high-order AR model (see ar.ols). The estimated residuals are then used for computing a least squares estimator of the full ARMA model. See Hannan and Rissanen (1982) for details. 
   

   The basic Hannan-Rissanen paper is in
   
   

   E. J. Hannan and J. Rissanen (1982): Recursive Estimation of Mixed Autoregressive-Moving Average Order. Biometrika 69, 81–94.
   
   
   We sure would like a copy of this reference.
   
   

3. 
   Wikipedia page on Time Series Analysis.
   Wikipedia ARMA page
   
   We have already implemented the Levinson and the Burg algorithms, but for computing the acf and pacf values. We hope to start Python coding in July after reading the above references.
   
   
   
4. Peter Tessin has made available his Python library for time series. It is readable and we hope to expand on his work. Peter Tessin: Time Series.pdf