Stabilise the results of a simple DL model for binary class classification

I am training a simple DL model for binary class classification, with 2 blocks of these layers, followed by a softmax layer:

featureInputLayer -> fullyConnectedLayer -> layerNormalizationLayer -> reluLayer

For a fixed train, validation and test set, I am getting diverse results in accuracy, ranging from 45-85%. Kindly suggest how to stabilise the results.

Here, training instances are 100 (50 per class), validation 24 (10 per class) and using

options = trainingOptions('adam', ...

'MaxEpochs', 100, ... % Adjust based on dataset size

'MiniBatchSize', 16, ... % Choose batch size based on memory

'ValidationData', valDS, ... % Validation data

'ValidationFrequency', 5, ... % Adjust based on dataset size

'Verbose', true, ...

'Plots', 'training-progress', ...

'OutputNetwork', 'best-validation-loss', ... % Save best model

'Shuffle', 'every-epoch',...

'LearnRateSchedule', 'cosine');

Answer 1

[Shantanu Dixit]

Hi Snigdha,

Given the small dataset (only 100 samples), the model's performance can vary a lot just due to random initialization. Getting more training data would help the most. If that's not possible, you can try augmenting the existing data to artificially increase the sample size. You can refer to 'imagedataaugmenter': https://www.mathworks.com/help/deeplearning/ref/imagedataaugmenter.html for detailed information.

Since the accuracy fluctuates so much (45-85%), it might also help to simplify the model a bit. Right now, with two dense blocks, it could be overcomplicating things for such a small dataset. You can try just one block first and see if the results become more stable.

Also, setting a fixed random seed "rng(42)" before training should make the results more repeatable between runs. If you're still seeing big swings, adding dropout layers: https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.dropoutlayer.html or reducing the learning rate could smooth things out.

Hope this helps!

[Snigdha]

Thank you for your response.

Meanwhile I have attempted the following:

1. Added dropout layer

2. Data Augmentation using SMOTE since my data is a feature vector

3. Changed batch size

4. Added and dropped layers also

But in all combinations of experiments, results are varying from 45-85%.

Kindly suggest.