Each regression function has a specific operation. This section
shows how to use these functions to perform specific types of regressions.
To illustrate use of the functions for various regressions, “typical”
usage is shown with optional arguments kept to a minimum. For a typical
regression, you estimate model parameters and residual covariance
matrices with the `mle`

functions and estimate the
standard errors of model parameters with the `std`

functions.
The regressions “without missing data” essentially ignore
samples with any missing values, and the regressions “with
missing data” ignore samples with every value missing.

Multivariate normal regression, or MVNR, is the “standard” implementation of the regression functions in Financial Toolbox™ software.

Estimate Parameters

[Parameters, Covariance] = mvnrmle(Data, Design);

Estimate Standard Errors

StdParameters = mvnrstd(Data, Design, Covariance);

Estimate Parameters

[Parameters, Covariance] = ecmmvnrmle(Data, Design);

Estimate Standard Errors

StdParameters = ecmmvnrstd(Data, Design, Covariance);

Least-squares regression, or LSR, sometimes called ordinary least-squares or multiple linear regression, is the simplest linear regression model. It also enjoys the property that, independent of the underlying distribution, it is a best linear unbiased estimator (BLUE).

Given *m* = `NumSamples`

observations,
the typical least-squares regression model seeks to minimize the objective
function

$$\sum _{k=1}^{m}{\left({Z}_{k}-{H}_{k}b\right)}^{T}\left({Z}_{k}-{H}_{k}b\right)},$$

which, within the maximum likelihood framework of the multivariate
normal regression routine `mvnrmle`

, is equivalent
to a single-iteration estimation of just the parameters to obtain `Parameters`

with
the initial covariance matrix `Covariance`

held fixed
as the identity matrix. In the case of missing data, however, the
internal algorithm to handle missing data requires a separate routine `ecmlsrmle`

to
do least-squares instead of multivariate normal regression.

Estimate Parameters

[Parameters, Covariance] = mvnrmle(Data, Design, 1);

Estimate Standard Errors

StdParameters = mvnrstd(Data, Design, Covariance);

Estimate Parameters

[Parameters, Covariance] = ecmlsrmle(Data, Design);

Estimate Standard Errors

StdParameters = ecmmvnrstd(Data, Design, Covariance);

Given* m *= `NUMSAMPLES`

observations,
the typical covariance-weighted least squares, or CWLS, regression
model seeks to minimize the objective function

$$\sum _{k=1}^{m}{\left({Z}_{k}-{H}_{k}b\right)}^{T}{C}_{0}\left({Z}_{k}-{H}_{k}b\right)$$

with fixed covariance *C*_{0}.

In most cases, *C*_{0} is
a diagonal matrix. The inverse matrix $$W={C}_{0}^{-1}$$ has diagonal elements that can
be considered relative “weights” for each series. Thus,
CWLS is a form of weighted least squares with the weights applied
across series.

Estimate Parameters

[Parameters, Covariance] = mvnrmle(Data, Design, 1, [], [], [], Covar0);

Estimate Standard Errors

StdParameters = mvnrstd(Data, Design, Covariance);

Estimate Parameters

[Parameters, Covariance] = ecmlsrmle(Data, Design, [], [], [], [], Covar0);

Estimate Standard Errors

StdParameters = ecmmvnrstd(Data, Design, Covariance);

An ad hoc form of least squares that has surprisingly good properties for misspecified or nonnormal models is known as feasible generalized least squares, or FGLS. The basic procedure is to do least-squares regression and then to do covariance-weighted least-squares regression with the resultant residual covariance from the first regression.

Estimate Parameters

[Parameters, Covariance] = mvnrmle(Data, Design, 2, 0, 0);

or (to illustrate the FGLS process explicitly)

[Parameters, Covar0] = mvnrmle(Data, Design, 1); [Parameters, Covariance] = mvnrmle(Data, Design, 1, [], [], [], Covar0);

Estimate Standard Errors

StdParameters = mvnrstd(Data, Design, Covariance);

Estimate Parameters

[Parameters, Covar0] = ecmlsrmle(Data, Design); [Parameters, Covariance] = ecmlsrmle(Data, Design, [], [], [], [], Covar0);

Estimate Standard Errors

StdParameters = ecmmvnrstd(Data, Design, Covariance);

Given a multivariate normal regression model in standard form
with a `Data`

matrix and a `Design`

array,
it is possible to convert the problem into a seemingly unrelated regression
(SUR) problem by a simple transformation of the `Design`

array.
The main idea of SUR is that instead of having a common parameter
vector over all data series, you have a separate parameter vector
associated with each separate series or with distinct groups of series
that, nevertheless, share a common residual covariance. It is this
ability to aggregate and disaggregate series and to perform comparative
tests on each design that is the power of SUR.

To make the transformation, use the function `convert2sur`

, which converts a standard-form
design array into an equivalent design array to do SUR with a specified
mapping of the series into `NUMGROUPS`

groups. The
regression functions are used in the usual manner, but with the SUR
design array instead of the original design array. Instead of having `NUMPARAMS`

elements,
the SUR output parameter vector has `NUMGROUPS`

of
stacked parameter estimates, where the first `NUMPARAMS`

elements
of `Parameters`

contain parameter estimates associated
with the first group of series, the next `NUMPARAMS`

elements
of `Parameters`

contain parameter estimates associated
with the second group of series, and so on. If the model has only
one series, for example, `NUMSERIES`

= 1, then the
SUR design array is the same as the original design array since SUR
requires two or more series to generate distinct parameter estimates.

Given `NUMPARAMS`

parameters and `NUMGROUPS`

groups
with a parameter vector (`Parameters`

) with ```
NUMGROUPS
* NUMPARAMS
```

elements from any of the regression routines,
the following MATLAB^{®} code fragment shows how to print a table
of SUR parameter estimates with rows that correspond to each parameter
and columns that correspond to each group or series:

fprintf(1,'Seemingly Unrelated Regression Parameter Estimates\n'); fprintf(1,' %7s ',' '); fprintf(1,' Group(%3d) ',1:NumGroups); fprintf(1,'\n'); for i = 1:NumParams fprintf(1,' %7d ',i); ii = i; for j = 1:NumGroups fprintf(1,'%12g ',Param(ii)); ii = ii + NumParams; end fprintf(1,'\n'); end fprintf(1,'\n');

Form an SUR Design

DesignSUR = convert2sur(Design, Group);

Estimate Parameters

[Parameters, Covariance] = mvnrmle(Data, DesignSUR);

Estimate Standard Errors

StdParameters = mvnrstd(Data, DesignSUR, Covariance);

Form a SUR Design

DesignSUR = convert2sur(Design, Group);

Estimate Parameters

[Parameters, Covariance] = ecmmvnrmle(Data, DesignSUR);

Estimate Standard Errors

StdParameters = ecmmvnrstd(Data, DesignSUR, Covariance);

Without missing data, you can estimate the mean of your `Data`

with
the function `mean`

and the covariance with the function `cov`

.
Nevertheless, the function `ecmnmle`

does this for
you if it detects an absence of missing values. Otherwise, it uses
the ECM algorithm to handle missing values.

Estimate Parameters

[Mean, Covariance] = ecmnmle(Data);

Estimate Standard Errors

StdMean = ecmnstd(Data, Mean, Covariance);

`convert2sur`

| `ecmlsrmle`

| `ecmlsrobj`

| `ecmmvnrfish`

| `ecmmvnrfish`

| `ecmmvnrmle`

| `ecmmvnrobj`

| `ecmmvnrstd`

| `ecmmvnrstd`

| `ecmnfish`

| `ecmnhess`

| `ecmninit`

| `ecmnmle`

| `ecmnobj`

| `ecmnstd`

| `mvnrfish`

| `mvnrmle`

| `mvnrobj`

| `mvnrstd`