Clear Filters
Clear Filters

How do you turn empty brackets ("[]") in a table into a NaN?

23 views (last 30 days)
Hi, I have a large dataset that I'm having some troubles with. There are a lot of areas in the dataset where there are missing datapoints. The issue is that some of the analyses I'm trying to run won't do so on this dataset unless the empty brackets are turned into NaNs.
I was trying to turn these empty brackets into NaNs using the isempty function, but on a table, it just doesn't work. Even if I call "isempty(dataset(i,j))" and i know it to be the empty brackets, MATLAB returns a logical 0, meaning no its not empty.
The only way I've been able to get isempty to work on the dataset is by turning my entire table into a cell. Unfortunately, that's just not feasible because then I'll lose all the variables in my table when i run "table2cell(x)". This is the script though:
dataset = Data_EMA(:,14); %Data_EMA is the name of the master dataset. I took out just one row from it to run this test script
for i = 1:height(dataset) %running script through i'th row, j'th column
if isempty(dataset{i,1})==1 %running script through i'th row, j'th column
dataset{i,1} = NaN; %if the i,j is an empty cell, it sets it to a NaN
end
end
The big issue here, as I've stated, is that I have to turn my table into a cell. Once I turn it back into a table, I'll lose all my variable names. I was thinking a solution here would be to write a giant for loop that runs through each column, saves the variable names, and then keeps concatenating each column to a new table and writes the old variable name back. This is obviously computationally expensive (going to take a very long time to run) so I'm wondering if anyone has any alterantive ways because I don't think its that big of an issue to where I would have to go down the route of using a loop that saves variable names, etc.
If someone has an idea, please let me know! I've attached the dataset to this post.
  2 Comments
Stephen23
Stephen23 on 19 Jan 2023
Edited: Stephen23 on 19 Jan 2023
"Even if I call "isempty(dataset(i,j))" and i know it to be the empty brackets, MATLAB returns a logical 0, meaning no its not empty."
For the simple reason that a scalar cell array is not empty:
S = load('Data_EMA.mat')
S = struct with fields:
Data_EMA: [3121×380 table] EMAData: [1×1 struct] ans: 1 dataset: {3121×1 cell} i: 3121
S.Data_EMA.Data_EMA14
ans = 3121×1 cell array
{[ 9]} {[ 6]} {[ 7]} {[ 7]} {[ 7]} {0×0 double} {[ 6]} {[ 8]} {0×0 double} {[ 7]} {[ 0]} {0×0 double} {[ 7]} {0×0 double} {[ 8]} {[ 2.5000]}
S.Data_EMA.Data_EMA14(6) % parentheses refer to the cell array itself.
ans = 1×1 cell array
{0×0 double}
Read what MATLAB is telling you: 1x1 cell array, it states very clearly. Is 1x1 empty? (hint: no).
However the content of a scalar cell array certainly might be empty:
S.Data_EMA.Data_EMA14{6} % curly braces returns the cell content
ans = []
The different indexing to refer to a cell array and its content is explained here:
BA
BA on 19 Jan 2023
Thanks, that's really helpful and appreciated!

Sign in to comment.

Accepted Answer

Walter Roberson
Walter Roberson on 19 Jan 2023
load Data_EMA;
mask = cellfun(@isempty,Data_EMA.Data_EMA14);
Data_EMA.Data_EMA14(1:10)
ans = 10×1 cell array
{[ 9]} {[ 6]} {[ 7]} {[ 7]} {[ 7]} {0×0 double} {[ 6]} {[ 8]} {0×0 double} {[ 7]}
Data_EMA.Data_EMA14(mask) = {NaN};
Data_EMA.Data_EMA14 = cell2mat(Data_EMA.Data_EMA14);
Data_EMA.Data_EMA14(1:10)
ans = 10×1
9 6 7 7 7 NaN 6 8 NaN 7
  8 Comments
Walter Roberson
Walter Roberson on 19 Jan 2023
When you originally read the data from the files, it would have been better if you had done
filename = 'AppropriateName.txt';
opts = detectImportOptions(filename);
opts = setvartype(opts, FIELDNUMBER, 'double');
table_for_this_file = readtable(filename, opts);
BA
BA on 19 Jan 2023
Thanks, I actually didn't know that. Will be using that from now on.

Sign in to comment.

More Answers (0)

Tags

Products


Release

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!