Deep learning regression network improvements?

Hi all.

I'm trying to train a deep learning model giving me as responses the curves I attached.

The skeleton of the simple model is attached, too.

How could I improve the network to obtain better results?

I'm wondering why the model is not able to reach the same order of magnitude of the training data (the attached images represents only few of the similar curves used for the training).

I tried increasing or reducing the size of the single layers, I tried to add more layers with a smaller size, but the result is still the same.

The input sequences are a representation of the initial conditions and wind velocity for each curve and it is recognizing the shape of the curves but not their magnitudes.

Thank you in advance.

Answer 1

[Mahesh Taparia]

Hi

By looking at your network design, it seem that you are having input data with dimension of 4 and trying to regress the output. You are taking the number of hidden nodes as 1000/2000 from 4 as input and converting back to 1 from 2000 nodes, which is not recomended. You can go with something like 4->8->16->32->16->8->1 (it deepend upon number of hidden layers you put). Try with different optimization techniques like adam/ sgdm, batch size, learning rate, number of hidden layers/nodes, input data normalization etc. Hope it will help.

[Giacomo Notaro]

Hi,

Thank you for the answer, in these days I worked to improve the model in the way you can see in the attached images.

I need to have the 4 inputs in order to define the initial conditions of the curves I'm trying to reproduce, in the same way I need only one output, hence I can't adopt the 4-8-16-32-16-8-4, since I need one sequence as output.

I improved the model by putting more networks in parallel as you can see in the picture attached, each column differs from the others by the learning rate. Which is coming from 1 of the first column to 6 at the last.

I will try to get better results trying to optimize also other features.

[Mahesh Taparia]

Yeah try different optimization techniques and it was some form of 4->8->16->32->16->8->1.

[Giacomo Notaro]

thank you!