addition of more variable that not exist in the CSV wile using readtable

Hello everyone,

I'm using readtable to read a large size CSV.

for some reasone the table is reading with an additional of 2 more columns (Var7 and Var8).

that happens even when I'm using detectimportoptions.

I'm not adding a variable at any place before, and in less heavier files the variables 7 and 8 are not exist in the table.

Is this a known error? anyone run into it and can advice how to avoid the unwanted variables?

[Simon Chan]

You may attach a small part of the file here so people is able to try.

[Walter Roberson]

If there are lines that end with commas, then they imply an extra (empty) variable.

[sani]

because the data file is large I cannot really preview it, neither in Excel or Matlab.

is there is a way to chack if this is the case?

Answer 1

[Star Strider]

One option is to use fileread or readcell to see if there is anything in those locations. Since most .csv files are actually Excel files (currently), using a format string as described in Create and Format Table from Text File and then suppressing the input of the last two columns by adding an asterisk to the format desciptor ('%*f%*f') is probably not possible.

Use the removevars function to remove them if necessary, as described in Delete Variables.

.

[Image Analyst]

Yes,

t = readtable('BadData.csv');
t = removevars(t, {'Var7', 'Var8'});

or, alternatively, setting to null might also work

if width(t) >= 7
    t(7:end) = [];
end

The latter will get rid of anything past column 6 so if you have 3 or 8 or 100 commas following one of the rows in your CSV file, it will get rid of them all.

[sani]

thank you, your solution is working for me :)

I'd really like to understand why is it happens only in very large CSV files

just a small correction:

if width(t) >= 7
    t(:,7:end) = [];
end

[Walter Roberson]

You do not need the if . You could just use

t(:,7:end) = [];

if the width is less than 7, such as if there are only 5 variables in the table, then it would be 7:5 which would be empty, and nothing would be selected for deletion.