How can I prepare proper input and output dataset for multilayer neural networks

I'm trying to create a neural network for the function y=x^2 with:

• 1 input layer
• 1 output layer
• 2 hidden layers, with each layer having 4 neurons

The code is as follows (Sample3.m):

% providing inputs and outputs  
x=-10:0.1:10;
y=x.*x;

% creating 2 hidden layers with 4 neurons each
hidden_layer_l2=[4 2];

% tansig=transfer function for layer 1
% logsig=transfer function for layer 2
% purelim=transfer function for output layer
network_l2=newff(x,hidden_layer_l2,{'tansig' 'logsig' 'purelin'},'trainlm');
view(network_l2);
net_train_l2=train(network_l2,x,y);

% simulate the data
simulated_data_l2=sim(net_train_l2,x);
plot(x,y,x,simulated_data_l2,'x');

The following error popped up:

Error using network/train (line 272)
Output data size does not match net.outputs{2}.size.
Error in Sample3 (line 40)
net_train_l2=train(network_l2,x,y);

I've browsed through the internet and came to know that I've to supply x*2 matrix to make it work, and currently I'm passing a vector. I'm new to matlab and neural networks and not sure how to make proper input set and output set (x and y).

Any help would be greatly appreciated !!

Answer 1

[Greg Heath]

One key to successful NN design is to use the simplest configuration possible. Typically, the higher the points to unknown weights ratio, the more accurate the design.

A basic theorem in NNs says that a single hidden layer is usually sufficient. Only when the number of hidden nodes becomes huge should you consider additional hidden layers with fewer total hidden nodes.

However, certain problems (typically involving images) are best solved in definable stages where each stage is performed by a single layer. CONVOLUTIONAL NNs are the best examples.

Since your problem does not fall into the latter category, just use 1 hidden layer and do not use more unknown weights than there are training equations to solve for those weights.

If

 [ I N ] = size(input)
 [ O N ] = size(target)

the number of UNKNOWN weights for a single hidden layer net with I-H-O node topology is

 Nw = (I+1)*H + (H+1)*O = O + (I+O+1)*H

The corresponding number of training equations is

 Ntrneq = Ntrn*O ~ 0.7*N*O

for the default 0.7/0.15/0.15 trn/val/tst data division ratios.

 For stable solutions the number of unknowns should not exceed the number of training equations. Therefore, it is desirable that

      Nw <= Ntrneq   <==> H <= Hub

where the H upper bound is given by

 Hub = (Ntrneq-O)/ (I + O +1)
     ~ (0.7*N-1)*O/(I+O+1)

Of utmost importance is that the stability of the solution increases as H/Hub DECREASES.

Therefore a good rule of thumb is to TRY to obtain solutions for H << Hub which I interpret as

           H <= ~ Hub/10

>> N = size(-10:0.1:10) % 201

 Hub = (0.7*201-1)*1/(1+1+1)  % 46.5667

Therefore choose Hmax = 46 and IF POSSIBLE,

      H < ~ 5 ~ Hmax/ 10

Therefore I would start with H = 5 and 10 different sets of initial random weights. If successful, reduce H; if not, increase H.

This approach is like cracking a peanut with a sledge hammer for this problem.

However, it is a general approach for problems that are extremely complicated.

Hope this helps.

Thank you for formally accepting my answer

Greg

[Akshay Joshi]

Thanks Greg !!