Neural Network - Multi Step Ahead Prediction Using NARX

I am modeling the network to predict the turning on and off the pump depending on the two variables. So the network is 2 inputs and 1 output with delay of 2 and hidden layer of 10. I have tried different combinations of transfer function, hidden layer and delays but the results are good enough. Output of pump is 0 or 1. Not sure if that is causing the problem.

Size(In)=Size(Out)=11,800

I have 5 inputs and 1 target. Is the data sufficient for the model?

I have tried with set of hiddenlayers and delays but not able to figure out what is the suitable number for current problem.

Is there any rule of thumb for for deciding these numbers for particular problem?

Here is the code:

X=tonndata(In,false,false);

T=tonndata(Out,false,false);

% % Input and target series are divided in two groups of data:

N = 50; % Multi-step ahead prediction

% 1st group: used to train the network

inputSeries  = X(1:end-N);

targetSeries = T(1:end-N);

% % 2nd group: this is the new data used for simulation. inputSeriesVal will 
be used for predicting new targets. targetSeriesVal will be used for network validation after prediction

inputSeriesVal  = X(end-N+1:end);

targetSeriesVal = T(end-N+1:end);

%% Choose a Training Function

trainFcn = 'trainlm';  % Levenberg-Marquardt

% trainFcn = 'trainbr';  % Bayesian regularization

%% Create a Nonlinear Autoregressive Network with External Input

delay=2;

inputDelays = 1:delay;

feedbackDelays = 1:delay;

hiddenLayerSize = 10;

net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize,'open',trainFcn);

%% Prepare the Data for Training and Simulation

[x,xi,ai,t] = preparets(net,inputSeries,{},targetSeries);

 % Setup Division of Data for Training, Validation, Testing*

net.divideParam.trainRatio = 70/100;

net.divideParam.valRatio = 15/100;

net.divideParam.testRatio = 15/100;

%% Train the Network

[net,tr] = train(net,x,t,xi,ai);

%% Test the Network

[Y,xf,af] = net(x,xi,ai);

e = gsubtract(t,Y);

performance = perform(net,t,Y)

%% Closed Loop Network

netc = closeloop(net);

netc.name = [net.name ' - Closed Loop'];

% view(netc)

[xc,xic,aic,tc] = preparets(netc,X,{},T);

yc = netc(xc,xic,aic);

closedLoopPerformance = perform(netc,tc,yc)

%% Step-Ahead Prediction Network

nets = removedelay(net);

nets.name = [net.name ' - Predict One Step Ahead'];

% view(nets)

[xs,xis,ais,ts] = preparets(nets,X,{},T);

ys = nets(xs,xis,ais);

stepAheadPerformance = perform(nets,ts,ys)

%% Prepare Input and Target Data for Testing the Network

[Xs1,Xio,Aio] = preparets(net,inputSeries(1:end-delay),{},targetSeries(1:end-delay));

[Y1,Xfo,Afo] = net(Xs1,Xio,Aio);

[netc,Xic,Aic] = closeloop(net,Xfo,Afo);

[yPred,Xfc,Afc] = netc(inputSeriesVal,Xic,Aic);

multiStepPerformance = perform(net,yPred,targetSeriesVal);

figure;

plot([cell2mat(targetSeries),nan(1,N);
      nan(1,length(targetSeries)),cell2mat(yPred);
      nan(1,length(targetSeries)),cell2mat(targetSeriesVal)]')

legend('Original Targets','Network Predictions','Expected Outputs')

Any comments and suggestions are very much appreciated.

Regards and thanks,

Abhishek

[Greg Heath]

 1. Format your code. Currently it is not understandable.
 2. size(In) = ? , size(Out) = ?
 3. Code is insufficient without data. Choose a 
MATLAB TIMESERIES EXAMPLE dataset:
    help nndatasets
    doc nndatasets
 4. Improper notation. You do not have a VAL set. You have a 
TEST set.
 5. CONFUSION: TRAINLM default has train, val and test subsets 
already defined in RANDOM ORDER!!!
 6. What are NMSEs and NMSEc (the OL and CL normalized MSE 
obtained by dividing with the corresponding target variance?)

 Take a good look at my tutorials in the NEWSGROUP.

Greg

[Abhishek Guhe]

I think this code only calculates MSE. With the below equations which I found in one of the Greg's answer will help in determining NMSE. I did not quite get the idea in point 6.

MSE = mse(e)

MSE00 = mean(var(e',1))

NMSE = MSE/MSE00 % Normalized MSE

I have updated the code.

Thanks Greg for suggestions.

[Greg Heath]

ERROR:

MSE00 is the mean of the TARGET variance, NOT the ERROR variance.

MSEtrn00, MSEval00 and MSEtst00 are the means of the training, validation and testing subset target variances.

NMSE = MSE/MSE00 is the fraction of target variance that is modeled by the net.

6DEC2016 CORRECTION:

NMSE = MSE/MSE00 is the fraction of target variance that is NOT modeled by the net.

Rsq = 1 - NMSE is the fraction of target variance that IS modeled by the net.

https://www.google.com/?gws_rd=ssl#q=wikipedia+r+squared

END OF 6JAN2016 CORRECTION

For typical nets 0 <= NMSE <= 1. Similar inequalities hold for NMSEtrn, NMSEval and NMSEtst.

The training subset estimate is optimistically biased because the same data is used to both estimate weights and evaluate the performance obtained from using those weights. This bias can be mitigated by adjusting the training performance estimates to take into account the number of degrees of freedom that are lost by using the same data for estimating parameters and performance.

 I   = input vector dimensionality
 O   = output vector dimensionality 
 NID = No. of input delays
 NFD = No. of feedback delays
 H   = No. of hidden nodes
 Nw = (NID*I+NFD*O+1)*H+(H+1)*O % No. of unknowns weights
 Ntrneq = Ntrn*O   % No. of training equations
 Ndof = Ntrneq-Nw % No. of degrees of freedom

 SSEtrn = sse(etrn)
 MSEtrn = SSEtrn/Ntrneq
 MSEtrna = SSEtrn/Ndof   % DOF "a"djusted

 MSEtrn00  = mean(var(ttrn,1))
 MSEtrn00a = mean(var(ttrn,0))

 NMSEtrn = MSEtrn/MSEtrn00
 NMSEtrna = MSEtrna/MSEtrn00a

For openloop training I favor the goal

 NMSEtrna <= 0.005

or

 MSEtrn <= 0.005*Ndof*MSEtrn00a/Ndof

Additional details can be obtained by searching the NEWSGROUP and ANSWERS using the above notation.

Hope this helps.

Greg

Answer 1

[Nick Hobbs]

An alternative to using the code for the neural network is to use the command 'nnstart'.

'nnstart' will launch a tool which would allow you to set options for your network more easily. You could then try various settings on the network to determine how it affects your results.

[Abhishek Guhe]

Thanks Nick for suggestion. I have exported the code from the nnstart and made changes for trying different combinations.