How do I loop over all columns in a cell array?

9 views (last 30 days)
lil brain
lil brain on 16 Mar 2023
Edited: Sulaymon Eshkabilov on 18 Mar 2023
The following code caluclates the outliers above 3 standard deviations for a single column in balls_data{2,1}(:,1).
data = balls_data{2,1};
mean_val = mean(cat(1, data{:})); % calculate the mean
std_val = std(cat(1, data{:})); % calculate the standard deviation
threshold = 2*std_val; % set the threshold for outlier detection
outliers = cellfun(@(x) x(x > mean_val + threshold | x < mean_val - threshold), data, 'UniformOutput', false); % find the outliers
I now need to run the same code over all columns in balls_data{2,1}.
Could someone help me make this into a loop?
Thanks!
  4 Comments
Show 2 older comments
Voss
Voss on 17 Mar 2023
Sounds like rmoutliers would work for that.
"B = rmoutliers(A) detects and removes outliers from the data in A.
  • If A is a matrix, then rmoutliers detects outliers in each column of A separately and removes the entire row."
data = cell2mat(balls_data{2,1});
data_no_outliers = rmoutliers(data,'mean');
Note that for rmoutliers(_,'mean'), "Outliers are defined as elements more than three standard deviations from the mean." And your code was using two standard deviations.
https://www.mathworks.com/help/matlab/ref/rmoutliers.html
lil brain
lil brain on 17 Mar 2023
Edited: lil brain on 17 Mar 2023
Interesting. This is much simpler of course! What if I would want to remove the outliers for each column in each of the 27 cells in balls_data while skipping over the empty cells?
I want to avoid using means or standard deviations for that matter whch are calculated from all columns in a cell or all cells in balls_data.
Could you be so kind to show me how that code would look?
Thanks!

Sign in to comment.

Sign in to answer this question.

Answers (3)

Voss
Voss on 17 Mar 2023
Ran in:
load('balls_data.mat')
balls_data is a cell array of cell arrays, except for two cells, which contain empty 0-by-0 numeric matrices
disp(balls_data)
{ 668×21 cell } {1195×21 cell } {1210×21 cell } {1911×21 cell } { 757×21 cell } { 520×21 cell } {1406×21 cell } {1090×21 cell } {1012×21 cell } { 572×21 cell } { 640×21 cell } {2655×21 cell } {1983×21 cell } { 513×21 cell } { 609×21 cell } {1654×21 cell } {1012×21 cell } { 564×21 cell } { 0×0 double} {1927×21 cell } { 941×21 cell } { 644×21 cell } { 748×21 cell } { 713×21 cell } { 662×21 cell } {2758×21 cell } { 0×0 double}
First, convert the cell arrays of scalars in balls_data to matrices, using cell2mat:
balls_data_mat = cellfun(@cell2mat,balls_data,'UniformOutput',false);
balls_data_mat is a cell array of matrices. Note that cell2mat([]) returns [], so the cells that contained the empty 0-by-0 matrices still contain them. But the other cells contain matrices now. So all cells contain matrices now.
disp(balls_data_mat)
{ 668×21 double} {1195×21 double} {1210×21 double} {1911×21 double} { 757×21 double} { 520×21 double} {1406×21 double} {1090×21 double} {1012×21 double} { 572×21 double} { 640×21 double} {2655×21 double} {1983×21 double} { 513×21 double} { 609×21 double} {1654×21 double} {1012×21 double} { 564×21 double} { 0×0 double} {1927×21 double} { 941×21 double} { 644×21 double} { 748×21 double} { 713×21 double} { 662×21 double} {2758×21 double} { 0×0 double}
Second, remove the rows containing outliers in each matrix:
balls_data_mat_no_outliers = cellfun(@(x)rmoutliers(x,'mean'),balls_data_mat,'UniformOutput',false);
balls_data_mat_no_outliers is a cell array of matrices. Now the matrices each have fewer rows than they did (except the empty ones, which remain empty), because the rows containing any outlier have been removed.
disp(balls_data_mat_no_outliers)
{ 617×21 double} {1169×21 double} {1210×21 double} {1722×21 double} { 757×21 double} { 469×21 double} {1227×21 double} {1042×21 double} {1004×21 double} { 529×21 double} { 609×21 double} {2370×21 double} {1983×21 double} { 466×21 double} { 585×21 double} {1548×21 double} { 969×21 double} { 555×21 double} { 0×0 double} {1828×21 double} { 905×21 double} { 614×21 double} { 666×21 double} { 706×21 double} { 598×21 double} {2706×21 double} { 0×0 double}
Sulaymon Eshkabilov
Sulaymon Eshkabilov on 16 Mar 2023
Here is the complete code to calculate all outliers:
load('balls_data.mat')
% Note balls_data is a cell array contains some empty cells as well. Thus,
% we need this step to remove them:
balls_data_0=balls_data(~cellfun('isempty',balls_data)); % Remove empty cells
for ii=1:length(balls_data_0)
data = balls_data_0{ii,1};
mean_val = mean(cat(1, data{:})); % calculate the mean
std_val = std(cat(1, data{:})); % calculate the standard deviation
threshold = 2*std_val; % set the threshold for outlier detection
outliers{ii,:} = cellfun(@(x) x(x > mean_val + threshold | x < mean_val - threshold), data, 'UniformOutput', false); % find the outliers
end
  4 Comments
Show 2 older comments
lil brain
lil brain on 17 Mar 2023
Hi @Sulaymon Eshkabilov,
thank you very much. However, when I run this code I get:
Assigning to 21 elements using a simple assignment statement is not supported. Consider using comma-separated list assignment.
Error in untitled5 (line 25)
outliers{ii,:} = cellfun(@(x) x(x > mean_val + threshold | x < mean_val - threshold), data, 'UniformOutput', false); % find the outliers
Why is that?

Sign in to comment.

Sulaymon Eshkabilov
Sulaymon Eshkabilov on 17 Mar 2023
Ran in:
Use this code as given here then you will not get the errors which are shown in your message thread:
load('balls_data.mat')
IDX =~cellfun('isempty',balls_data);
for ii=1:length(balls_data)
if IDX(ii)~=0
data = balls_data{ii,1};
mean_val = mean(cat(1, data{:})); % calculate the mean
std_val = std(cat(1, data{:})); % calculate the standard deviation
threshold = 2*std_val; % set the threshold for outlier detection
outliers{ii,:} = cellfun(@(x) x(x > mean_val + threshold | x < mean_val - threshold), data, 'UniformOutput', false); % find the outliers
else
outliers{ii,:} = []; % Will contain an empty cell where balls_data is empty
end
end
size(outliers)
ans = 1×2
27 1
size(balls_data)
ans = 1×2
27 1
  2 Comments
lil brain
lil brain on 17 Mar 2023
Sure! But I still get the same error for some reason.
Assigning to 21 elements using a simple assignment statement is not supported. Consider using comma-separated list assignment.

Sign in to comment.

Categories

Find more on Problem-Based Optimization Setup in Help Center and File Exchange

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!