How do you code Garson's algorithm in matlab to find the relative importance of parameters when training a neural network model?

18 views (last 30 days)
Haris Saeed on 28 Dec 2023
Commented: Haris Saeed on 28 Dec 2023
I am working on a predictive analysis study and came across garson's algorithm but I am facing trouble in programming the formula for it. Here is my code:
%%% Data Inputs
inputs = data(:, 1:4)';
targets = data(:, 5)';
%%% Create a feedforward neural network
net = feedforwardnet([10, 15, 20]);
%%%% Set the number of neurons in each hidden layer
net.layers{1}.size = 10;
net.layers{2}.size = 15;
net.layers{3}.size = 20;
%%%% Change activation functions (e.g., 'tansig' to 'purelin' for output layer)
net.layers{1}.transferFcn = 'tansig'; % Activation function for first hidden layer
net.layers{2}.transferFcn = 'tansig'; % Activation function for second hidden layer
net.layers{3}.transferFcn = 'tansig'; % Activation function for third hidden layer
net.layers{4}.transferFcn = 'purelin'; % Activation function for output layer
%%%% Divide data for training, validation, and testing
net.divideParam.trainRatio = 0.7;
net.divideParam.valRatio = 0.15;
net.divideParam.testRatio = 0.15;
%%%% Train the neural network
[net, tr] = train(net, inputs, targets);
%%%% Calculate Garson's importance
importance = garsonsAlgorithm(net);
%%%% Display feature importances
disp('Feature Importances (Garson''s Algorithm):');
for i = 1:length(importance)
disp(['Importance for x' num2str(i) ': ' num2str(importance(i))]);
end
Here is the garson's algorithm function that I wrote down:
function importance = garsonsAlgorithm(net)
%%% Extract connection weights from the neural network
weights = cell2mat(net.IW);
% Check if there are bias weights
if isfield(net, 'b')
bias_weights = cell2mat(net.b);
weights = [weights; bias_weights];
end
%%% Calculate the absolute values of the weights
absolute_weights = abs(weights);
%%% Calculate the importance for each input feature
importance = sum(absolute_weights, 1);
%%% Normalize importance values
importance = importance / sum(importance);
end
I was wondering if this is correct or not since I am new to programming. Also, would there be any sort of variation of outputs (relative importance) when changing the number of inputs in the garson's algorithm?
I have attached the formula for Garson's algorithm for reference:

Debraj Maji on 28 Dec 2023
Edited: Debraj Maji on 28 Dec 2023
I understand that you are trying to code Garson's Algorithm in MATLAB. I have implemented the function below for your reference. Please note that this implementation assumes a single output neuron. If your network has multiple output neurons, you'll need to adjust the algorithm accordingly. The importance values are printed as percentages, indicating the relative importance of each input variable to the network's output.
function garsonsAlgorithm(net)
IW = net.IW{1}; % Input-to-hidden weights
LW = net.LW{2,1}; % Hidden-to-output weights
% Number of input and hidden neurons
numInputs = size(IW, 2);
numHidden = size(IW, 1);
% Calculate the connection strengths
hiddenLayerWeights = abs(IW);
outputLayerWeights = abs(LW);
% Calculate the relative importance (RI) for each input neuron to each hidden neuron
RI = hiddenLayerWeights ./ sum(hiddenLayerWeights, 2);
% Calculate the contribution of each hidden neuron to the output neuron
contribution = RI .* outputLayerWeights';
% Normalize contribution for each input to sum to 1
inputImportance = contribution ./ sum(contribution, 1);
% Display the importance of each input variable
for i = 1:numInputs
fprintf('Input %d importance: %.2f%%\n', i, 100 * inputImportance(i));
end
end
Given below is the code I used to test the function created above:
X = rand(10, 100); % 10 input features, 100 samples
T = rand(1, 100); % 1 output, 100 samples
% Create and train a feedforward neural network with one hidden layer
net = feedforwardnet(5); % 5 hidden neurons
net = train(net, X, T);
% Apply Garson's algorithm to the trained network
garsonsAlgorithm(net);
The results so obtained were as follows:
Secondly, changing the number of inputs to a neural network can affect the relative importance scores calculated by Garson's algorithm due to weight redistribution.
I hope this helps,
With regards,
Debraj.
Haris Saeed on 28 Dec 2023
Thank you mate for your response. This has solved a big problem for me

Categories

Find more on Define Shallow Neural Network Architectures in Help Center and File Exchange

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!