Missing rows in cell array generated from CSV file

I am using a MATLAB script that uses the readcell function to read in data from multiple CSV files. Each file is relatively large (4.5 MB) and is read in with the 'ConsecutiveDelimitersRule' set to 'split'.

When I compare the MATLAB cell generated by readcell to the original CSV file, several rows in the CSV file are not present in the cell array. There does not appear to be a consistent pattern of which rows are skipped over during the import process.

Are there any potential issues with this setup given the file format and size?

[Torsten]

Can you include a small .csv file and your code to reproduce this behaviour ?

[dpb]

As @Torsten says, it will not be possible to diagnose this without a file that creates the issue. If you can't simply snip out a section around the apparently missing section of the original file and reproduce the symptoms, you would be able to post a compressed version of the file if run into size limits. Of course, the smaller you can make and still reproduce the problem, the better...

My crystal ball says will find some aberration in the file construction underlying it, but "ya' never knows!"

[Stephen23]

"Are there any potential issues with this setup given the file format and size?"

Much more likely cause: the file content.

[Thomas]

I have done some more investigation, and it additionally appears that various rows in the input CSV file have their order changed in the cell array generated by readcell (e.g., a row that appears before another row in the CSV file appears after it in the cell array).

As a test, I tried importing the CSV file through the Import tool, and this method doesn't yield any missing/re-ordered data (all rows are present and in the correct order). Should these two methods yield the same output with no non-default settings in readcell?

Does this additional information point to any specific cause for this issue?

[Stephen23]

"Does this additional information point to any specific cause for this issue?"

Probably the file content.

Once you provide an MWE we can investigate.

[dpb]

Most likely causes are things like differing number of delimiters in one or more rows, embedded nonprinting characters, malformed or non-quoted strings and the like. It is, however, impossible to diagnose a precise issue here without an example file that exhibits the symptoms.

Answer 1

[Ishan]

A CSV file of this size should generally not cause readcell to omit or reorder rows. In particular, row reordering would not be expected from a sequential import operation.

Based on the behavior described, the issue is more likely related to how the file is being parsed than to the file size itself. Common causes include embedded delimiters, quoted text fields, line breaks within quoted fields, inconsistent field counts between rows, or other formatting irregularities that can affect CSV parsing.

Since the Import Tool appears to read the file correctly, one possible approach is to use:

opts = detectImportOptions(filename);
C = readcell(filename,opts);

This uses the same detected import configuration that the Import Tool would typically derive from the file.

If the results differ between the Import Tool and readcell, I would inspect the file for quoting and delimiter-related formatting issues, particularly near the rows that appear to be missing or reordered.