Replicating fitnet training performance with trainNetwork

I am trying to design my own deep neural hidden network to approximate a functional relationship between an input vector and one target variable but I fail to obtain good results when designing my own deep neural network and I would like to understand the reason. The fit is pretty much perfect when I use the fitnet & train functions. To narrow down my problem, I brought it down to a one input, one target problem. The total number of samples is 2000.

The actual relationship looks as follows (input and output variables are already mapped to [-1,1]):

Fitnet / train

Using fitnet, I achieve almost arbitrary accuracy using the following options to create a NN with two hidden layers with ten nodes each:

trainFcn = 'trainlm';
hiddenLayerSize = [10 10];
net = fitnet(hiddenLayerSize,trainFcn);
net.trainParam.min_grad=1e-10;
net.trainParam.max_fail = 500;
net.trainParam.epochs=5000;
net.divideFcn = 'dividerand';
net.divideMode = 'sample';
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
net.performFcn = 'mse';
[net,tr] = train(net,x,t, 'useparallel', 'yes');

In this case, the training only takes a couple of seconds and the resultant fit is pretty much perfect:

trainNetwork

However, I fail to obtain a good fit when designing my own deep neural network (I am using an imageInputLayer as suggested in another post)

layers = [imageInputLayer([1 1 1]);
fullyConnectedLayer(10)
tanhLayer
fullyConnectedLayer(10)
tanhLayer
fullyConnectedLayer(1);
regressionLayer];
options = trainingOptions('sgdm', 'maxepochs', 10000, 'minibatchsize', 2000, 'InitialLearnrate', 0.1,'Plots','training-progress');
XNew = reshape(x, [1,1,size(x,1),size(x,2)]); % size(XNew) =  [1, 1, 1, 2000]
trainedNet = trainNetwork(XNew, t', layers, options);

I tried playing around with the Initial learning rate (from 0.1 to 1e-5), the minibatchsize (from 10 to all samples), the solver and the activation functions but after a fast decrease the RMSE always gets stuck around 0.1:

and this is the fit I get:

Am I doing anything fundametally wrong or does anyone have tips on how to improve the learning?

Thanks a lot in advance!

[Steen Sorensen]

(Sorry for the revival of this old post)

did you ever find an answer to this question? I am in a very similar situation.

Thank you in advance

[Yifan Wei]

I have faced same issue here. did you get a chance to look at why this happened?

[ytzhak goussha]

You should try to run this without normalization in the imageInputLayer since you already scaled it [-1 1].

The defult is normalization be zero centering by subtracting the mean, the mean is a learnable parameter.

Try:

imageInputLayer([1 1 1],'Name','input','Normalization',"none")