svm and sentimental analysis

Hi,

I am a newbie to coding.

I get the error saying

"Error using horzcat

Dimensions of arrays being concatenated are not consistent.

Error in svmsent (line 44)

XTrain = [XTrain, sentimentScoresTrain];"

How should I fix it?

filename = "sentiment_irrelevantdrop_Chegg"; 
data = readtable(filename,'TextType','string');
data.sentiment = categorical(data.sentiment);

% Split dataset into training and test sets using holdout
cvp = cvpartition(data.sentiment, 'Holdout', 0.1);
dataTrain = data(cvp.training, :);
dataTest = data(cvp.test, :);

% Extract review text and sentiment labels from training and test set 
textDataTrain = dataTrain.text;
textDataTest = dataTest.text;
YTrain = dataTrain.sentiment;
YTest = dataTest.sentiment;

% Preprocess training set
documents = preprocessText(textDataTrain);

% Create bag of words and remove infrequent words
bag = bagOfWords(documents);
bag = removeInfrequentWords(bag,2);
[bag,idx] = removeEmptyDocuments(bag);
YTrain(idx) = [];

% Encode training set using bag of words
XTrain = bag.Counts;

% Train SVM classifier
mdl = fitcecoc(XTrain, YTrain, "Learners", "linear");  

% Preprocess test set
documentsTest = preprocessText(textDataTest);
documentsTrain = preprocessText(textDataTrain);

% Encode test set using bag of words
XTest = encode(bag, documentsTest);

% Compute sentiment scores for training and test sets using VADER
sentimentScoresTrain = vaderSentimentScores(documentsTrain);
sentimentScoresTest = vaderSentimentScores(documentsTest);

% Concatenate sentiment scores with bag of words features
XTrain = [XTrain, sentimentScoresTrain];
XTest = [XTest, sentimentScoresTest];

% Build new svm model using both bag-of-words and vader sentiment scores as
% features
mdl2 = fitcecoc(XTrain, YTrain, "Learners", "linear");  

% Predict sentiment labels for test set
YPred = predict(mdl, XTest);

% Evaluate performance
accuracy = sum(YPred == YTest) / numel(YTest);
fprintf("Accuracy: %.2f%%\n", accuracy * 100);

confusion = confusionmat(YTest, YPred);
truePositive = confusion(1, 1);
falsePositive = confusion(2, 1);
trueNegative = confusion(2, 2);
falseNegative = confusion(1, 2);

% Compute precision, recall, and F-measure
precision = truePositive / (truePositive + falsePositive);
recall = truePositive / (truePositive + falseNegative);
fMeasure = 2 * precision * recall / (precision + recall);

% Compute accuracy 
accuracy2 = (truePositive + trueNegative) / numel(YTest);

% Display results 
disp(['True positive: ' num2str(truePositive)]);
disp(['False positive: ' num2str(falsePositive)]);
disp(['True negative: ' num2str(trueNegative)]);
disp(['False negative: ' num2str(falseNegative)]);
disp(['Precision: ' num2str(precision)]);
disp(['Recall: ' num2str(recall)]);
disp(['F-measure: ' num2str(fMeasure)]);

function documents = preprocessText(textData)
documents = tokenizedDocument(textData);
documents = addPartOfSpeechDetails(documents);
documents = removeStopWords(documents);
documents = erasePunctuation(documents);
documents = removeShortWords(documents,2);
documents = removeLongWords(documents,15);
end

Answer 1

[Manikanta Aditya]

Hi Sanguk,

As per my understanding, you would like to know how to fix the error -‘Dimensions of arrays being concatenated are not consistent’ in your code.

The error message suggests that the dimensions of ‘XTrain’ and ‘sentimentScoresTrain’ do not match and therefore cannot be concatenated using ‘horzcat’. ‘XTrain’ and ‘sentimentScoresTrain’ should have same number of rows before concatenating them. You can check the size of ‘XTrain’ and ‘sentimentScoreTrain’ using the ‘size’ function.

size(XTrain)
size(sentimentScores)

Alternatively, you can concatenate vertically using the ‘vertcat’ function.

XTrain = vertcat(XTrain, sentimentScoresTrain);

This will add the ‘sentimentScores’ as additional rows to the ‘XTrain’ matrix, rather than as additional columns.

For further reference, please check this MATLAB Answer:

• https://www.mathworks.com/matlabcentral/answers/370972-dimensions-of-matrices-being-concatenated-are-not-consistent

Please refer to the following documentation for more info on

• ‘vertcat’ function: https://www.mathworks.com/help/matlab/ref/double.vertcat.html
• ‘size’ function: https://www.mathworks.com/help/matlab/ref/size.html

I hope this resolves the issue you were facing.