NARX how to control batch sequences

Hello,

I would be glad if you could help me with the following.

I wanted to use a NARX network approximate a function with one out put and 6 inputs. For this I have 200 Matrices where the function values change over time (each aprox. 20000 Timesteps).

I thought it would be most practical to train the network with one such Matrice per batch because the Matrice is representing a "one lifetime" and different Matrices differ in the usage of the product. The output depends on the history of the 6 Inputs.

So far I have tried it with Array (1x200 Matrices = different usage) each Matrice (6 Inputs x 20000 Timesteps)

and

Array (1x20000 Matrices = Timesteps) each Matrice (6 Inputs x 200 Matrices = different usage)

Both times the performance was that bad that I could not guess which way is the right one and if it is possible anyway. I thought I saw somewhere that one Cell of an Array Input is used as one batch in the default training.

Thank you very much

Dominic

[Greg Heath]

Sorry. I don't understand.

What is the product?

What are the 6 inputs and 1 output?

You nave 200 input matrices of size 6x20000 and 200 corresponding 1x20000 output vectors?

Do you really need to predict future outputs? Current outputs are not sufficient? How far into the future do you need to predict?

Are the 7 variables stationary ?

If you divide each matrix into 20 blocks of 1000, are the mean, variance and significant auto and cross correlations the same?

You have so much data, you really don't need a validation set.

Greg

P.S> Do not use the default data division function 'dividerand'

[Dominic]

Thanks Greg for the answer.

First I made a mistake. I got 2000 Timesteps per Cell. Let me clarify my Data.

To tell at which state of lifetime a product is I got 6 different variables I can observe. Thus these are the 6 inputs. The state of lifecycle is the output and ranges from 0 to 1. When the product reaches 0 it needs to be repaired and the whole things starts again, but of course during the lifecycles the product ages and thus the behaviour changes slightly. Now the typical lifecycle is about 2 000 Timesteps long and I got data from 200 different lifecycles during the lifetime of one product.

But again I would like to see the variation between the lifecycles as variations of slightly different behaviour and I would like to get the state of the lifecycle as output.

I do not need to forecast some future values. That is right. But I saw it being done with similar problems and it lead to a better outcome. The problem is that my Inputs plotted over time are very noisy and the output over time is a stable set of (almost)linear parts.

I guess the fact that the input data is highly dynamic reflects in the variables not being stationary over time. I just looked it up and found i10test as function to test the stationarity of variables in Matlab.

So I will post a result here in the next comment.

Thank you already for your help.

In general I can live with the fact that the method with NARX does not lead to proper results... but in order to get to this point I need to be sure that I used the toolboxes right.

P.S. Thanks for your data division function trick. I am already using divideblock like you mentioned in your many helpfull answers to other questions.

[Dominic]

Test Results

        I(1)    I(0)    
======================
var1    0       1       
        0.8575  0.0100  
----------------------
var2    1       1       
        0.0010  0.0100  
----------------------
var3    0       1       
        0.6118  0.0100  
----------------------
var4    0       1       
        0.0505  0.0100  
----------------------
var5    1       1       
        0.0010  0.0100  
----------------------
var6    0       1       
        0.9990  0.0100  
----------------------
var7    0       1       
        0.9990  0.0100  
----------------------

The i10test - function gave me the following results, where var7 is the target and var1 to var6 are the inputs. Var1 by the way is the time within one lifecycle.

[Dominic]

I attached plots of exemplary input and target data and the relationship between the Matrices. Input5 and Input6 are calculated(simple Product and Integration) from the other 4 Inputs and in the best case I want to get rid of them.

In subplot 8 and 9 you see the Input3 and Target throughout all Matrices.

So I already grouped repeating patterns from the orignal data in the Matrices and each cell is such a pattern. But of course they still vary.

[Greg Heath]

For each variable you have a 2x2 matrix. What does each number mean?

Have you calculated the frequency spectrum of each output signal?

Are the spectra consistent? What feedback time delays are indicated from the dominant peaks at positive frequencies? Are they the same as indicated in the autocorrelation functions?

If so, what are the significant positive lags?

Are the cross spectra of inputs and outputs consistent? What input time delays are indicated from the dominant peaks at nonnegative frequencies? Are they the same as indicated in the crosscorrelation functions?

If so, what are the significant nonnegative lags?

If lag = 0 is significant, try fitnet before designing timeseries.

It may be prudent to design timedelaynet and narnet before designing narxnet.

Good Luck

[Dominic]

Thanks again for your answer Greg. I got some question regarding your suggested approach:

For each variable you have a 2x2 matrix. What does each number mean? If you are speaking about the output of the i10test function the matlab help tells me "displays the results of paired integration and stationarity tests on the variables in" so I thought that is the answer to your quesition about the stationarity.

Have you calculated the frequency spectrum of each output signal? I guess best way to do this is with fft() right?

Are the spectra consistent? I guess you mean having peaks at the same frequencies right?

What feedback time delays are indicated from the dominant peaks at positive frequencies? There are two pretty huge peaks one at 0 the other one at 7,683e+05

Are they the same as indicated in the autocorrelation functions? Here I assume the best way to get it is the crosscorr fcn I wasn't sure whether you meant to calculate the crosscorr from the timeseries or the spectrum. So left one is spectrum and right one is timeseries.

If so, what are the significant positive lags? For the time series I guess the plot need to be wider to estimate that since all lags are significant aren't they?

Are the cross spectra of inputs and outputs consistent?

No

crosscorr(timeseriesInput,timeseriesOutput)

• What input time delays are indicated from the dominant peaks at nonnegative frequencies? Are they the same as indicated in the crosscorrelation functions?*

Can you tell me how to get those?

If so, what are the significant nonnegative lags?

If lag = 0 is significant, try fitnet before designing timeseries.

I tried fitnet and the feedforwardnet already both with quite poor performance and lots of noise on the output. But is my assumption right that with those both you can group batches by putting them in one cell?

It may be prudent to design timedelaynet and narnet before designing narxnet. I thought the NARX net is the most basic one and the two other are special versions?

Good Luck

Thanks again for your answer

[Greg Heath]

When I look at your plots I notice the following

Something very significant happens ~ t= 1800

var5 = var2

target = var6

Is that consistent with other data sets?

[Dominic]

var5 = var2

As described above var5 is the product of var2 and var3, and var6 is the integration of var2. Both inputs are only supposed to be used in a first stage because they put additional requirements on the observation.

The goal is clearly to have the network giving a more or less accurate output from getting part of the timeseries.

About the ~t = 1800 yes it is the more or less the same thing with the other matrices. There the product is recharged/refilled before it is drained again.

Thank you very much.