Is there a Row limit when transfering a matlab table to CSV file?

I have a MATLAB table of 8 columns and 246 312 ROWS. I tried to transfer the MATLAB table to CSV in order to be able to use it in STATA cause I have a MAC and the xlwrite or any other function to transfer to an excel file from MATLAB sends an error mess. So i am using the matlab function:

 >> writetable(A, 'filename.csv', 'Delimiter',' ,' ,'QuoteStrings', true).

I get a CSV file with all the 8 columns but only a part of the rows, i get 139 521 instead of the 246 312 rows.

WHY???

How can i get all the rows in?

I tried transferring the CSV file to EXCEl to double check if excel was gonna pick it up, but EXCEL too only gets 139 521.

How can i do to correct that?

PS: My EXCEL version is a MAC version 2011

Thank you for your help.

Answer 1

[Walter Roberson]

I tested with

A = array2table( randi([0 255], 246312,8,'uint8') );
writetable(A, 'filename.csv', 'Delimiter',',' ,'QuoteStrings', true)

Note that I had to modify the delimiter from ' ,' to ',' alone as writetable only permits a single character for the delimiter (unless a \ escape is used, and even then it has to format into a single character.)

>> !wc filename.csv
  246313  246313 7035422 filename.csv

So it got the right number of lines (the extra one is the header)

I opened it with Microsoft Office 2011 on my Mac; all of the lines were there.

[Amira Dabchy]

Got it!!! Thank you so much!! It works and I got all the rows

Answer 2

[Massimo Zanetti]

Depending on Office version, there are limitations in the maximum number of rows Excel can load. To really know how many rows the CSV file has, open it with another software that does not have limitations. For example: http://www.delimitware.com/

[Amira Dabchy]

Thank you for your answer! Yes i checked the CSV file and it is also missing lots of rows. It has the same number of rows as the excel sheet, which is 139 521 instead of 246 312 ROWS. So the problem is coming from the CSV file. How can i correct that? I dont see what i did wrong using writeable(...).

Thank you!

[Massimo Zanetti]

I suggest use a specific function for csv writing: https://it.mathworks.com/help/matlab/ref/csvwrite.html

If this answer helped you, please accept it. :)

[Amira Dabchy]

Yes i actually already did but all I get is an empty ".dat" file and an error mess. When i try to open this empty file i get an error mess:

Here is the error mess:

>> csvwrite('STATA.dat',STATA);
Undefined function 'real' for input arguments of type 'table'.

Error in dlmwrite (line 189)
                    str = sprintf('%.*g%+.*gi',precn,real(m(i,j)),precn,imag(m(i,j)));

Error in csvwrite (line 42)
dlmwrite(filename, m, ',', r, c);

I also tried it with the function dlmwrite and its not working.

Am i doing it wrong?

Is there any other way to get the Matlab data table that i have into STATA? (other than transferring directly to excel which does not work on a MAC)

[Massimo Zanetti]

Type "help real" and then press Enter in your command line. What is the output?

[Walter Roberson]

csvwrite() is not defined for table() data objects, and it happens that the code does not have a specific check to protect against table() objects so it happens that the call to real() is the place the code fails. It could have been almost anything; do not worry about trying to chase down real() .

[Massimo Zanetti]

This shouldn't be the bottleneck.. indeed one can convert table object into array using table2array(), then use csvwrite, which I still believe is more appropriate.

[Walter Roberson]

table2array is only for tables that are purely numeric. The fact that the code uses 'QuoteStrings', true suggests that at least some of the entries are strings rather than numeric.

table2cell() followed by xlswrite() might be appropriate.

[Massimo Zanetti]

Yes, you are right. But, as far as Amira says xlswrite didn't work.. That is way I proposed another option (which is indeed the Matlab built-in function to write CSV).

[Walter Roberson]

xlwrite() from the File Exchange and xlswrite() from MATLAB are not defined in table() data objects.

xlswrite() of a cell containing strings to a .csv file is not supported in Basic mode, which the user would be restricted to because they are on a Mac.

table2cell() followed by xlwrite() from the File Exchange might work. (I do not have the poi_library installed at the moment so I cannot tells at this time.)