Cascade-Correlation Neural Networks

This topic explains cascade-correlation neural networks, their advantages, and how you can use them to identify nonlinear ARX models.

What Are Cascade-Correlation Neural Networks?

A feedforward neural network consists of a series of layers where the first layer has a connection from the network input and each subsequent layer has a connection from the previous layer. The final layer produces the network output. A cascade-forward neural network includes a connection from the input and every previous layer to the following layers. When you create either type of network, you have to guess the network architecture and specify it in advance, and you cannot modify the number of layers and number of neurons in each layer after you create it.

On the other hand, hidden units in a cascade-correlation neural network are added to the network one at a time and frozen before more units are added. Each hidden unit is trained to maximize the correlation between its output and the network residual error. The network includes a connection from all hidden units to all previous layers and all inputs. You do not have to specify the network architecture in advance.

Advantages of Cascade-Correlation Neural Networks

Cascade-correlation neural networks have these advantages when compared to feedforward and cascade-forward networks:

• You do not have to guess the network architecture in advance. The cascade-correlation neural network determines the number of layers in the network. Each of these layers has one unit. The network accounts for network optimality or near-optimality.

• The network has a connection from all units to all previous layers and inputs. This extensive connectivity has proven to perform well in modeling dynamic system behavior.

Using Cascade-Correlation Neural Networks in Nonlinear ARX Models

Using a neural network as part of the nonlinear ARX model enhances its capability to approximate any arbitrary nonlinear dynamics.

A nonlinear ARX model consists of model regressors and an output function. In the simplest case, regressors are delayed inputs and outputs. The output function contains one or more mapping objects, one for each model output. Each mapping object can include a linear and a nonlinear function that act on the model regressors to give the model output and a fixed offset for that output. This block diagram represents the structure of a single-output nonlinear ARX model in a simulation scenario.

For more information on the structure of nonlinear ARX models, see What Are Nonlinear ARX Models?

The System Identification Toolbox™ software provides several flexible nonlinear functions as mapping objects for nonlinear ARX models, such as wavelet and sigmoid networks. For a list of mapping functions, see Available Mapping Functions for Nonlinear ARX Models.

You can specify a cascade-correlation neural network as the mapping function by using an idNeuralNetwork object with "cascade-correlation" as the first input argument. To reserve a sample of data for cross-validation during training of the nonlinear ARX model, configure the CrossValidate and CrossValidationOptions properties of the nlarxOptions option set. You can then estimate the model using the nlarx function.

This block diagram shows the cascade-correlation neural network replacing the linear and nonlinear functions as the output function for a single-output nonlinear ARX model.

A cascade-correlation neural network is a static network. So, to estimate a dynamic model, you need to use regressors as inputs.

References

[1] Fahlman, Scott, and Christian Lebiere. "The Cascade-Correlation Learning Architecture." Advances in Neural Information Processing Systems 2 (1989).