Problem by using str2double with csv file

i have some csv files and i extract some rows from them using readtable (i tried also readmatrix). when i used str2double to the extracted vectors, it gives wrong result ( with e+16 and e+5). Have please anyone help me ?

[Adam Danz]

How is " e+16 and e+5" a wrong result? What result were you expecting?

Try to provide all of the relevant information so we can fully undestand the problem.

[Ahmed Ghouma]

this is an example of code

T15 = readmatrix('file.csv');

a15=T15(:,2); %%this what i have in the csv file [-0,000179329035810182 -0,000174311849256171 -0,000165976742269641 ]

x15=str2double(a15);%% it gives [ -1.74311849256171e+16 -1.65976742269641e+15 -1.69522570952220e+16]

b15=T15(:,3); %% this what i have in the csv file [2,77364727348772E-06 3,49E-05 3,06E-05 ]

w15=str2double(b15); %% it gives [27736472734.8772 349462499135.450 305549772572.879 ]

[Scott MacKenzie]

It might help if you post the data file.

[Ahmed Ghouma]

this is the data file

[Scott MacKenzie]

Well, that file might have csv as the suffix but it does not contain comma-separated values. There are no commas in the file.

[Ahmed Ghouma]

semicolon is the delimiter ";" and what should be done ?

when i use xlsread it gives NaN

Answer 1

[the cyclist]

The issue is that your input file uses commas as the decimal separator, rather than a period ("decimal point"). This means that MATLAB is treating those elements a text rather than numbers.

I did a global replacement of commas for periods, and then your file loads the data as numbers.

T15 = readtable('73_4.csv');

Warning: Column headers from the file were modified to make them valid MATLAB identifiers before creating variable names for the table. The original column headers are saved in the VariableDescriptions property.
Set 'VariableNamingRule' to 'preserve' to use the original column headers as table variable names.

output = T15(1:3,2)

output = 3×1 table
    Trace1_Gain_Real
    ________________

      -0.00011834   
      -0.00012968   
      -0.00017933   

I don't think there is a way to change MATLAB's requirement of using a decimal point for input. So, I think that doing the replacement in the input file is your only option. (But I could be wrong.)

[Stephen23]

"I don't think there is a way to change MATLAB's requirement of using a decimal point for input. "

T = readtable('73_4.csv', 'Delimiter',';', 'DecimalSeparator',',')

Warning: Column headers from the file were modified to make them valid MATLAB identifiers before creating variable names for the table. The original column headers are saved in the VariableDescriptions property.
Set 'VariableNamingRule' to 'preserve' to use the original column headers as table variable names.

T = 201×7 table
    Frequency_Hz_    Trace1_Gain_Real    Trace1_Gain_Imaginary    Trace1_Gain_Magnitude_dB_    Trace2_Gain_Real    Trace2_Gain_Imaginary    Trace2_Gain_Phase___
    _____________    ________________    _____________________    _________________________    ________________    _____________________    ____________________

          1e+06        -0.00011834            2.7736e-06                   -78.535               -0.00011834            2.7736e-06                 178.66       
      1.145e+06        -0.00012968            3.4946e-05                   -77.438               -0.00012968            3.4946e-05                 164.92       
       1.29e+06        -0.00017933            3.0555e-05                   -74.803               -0.00017933            3.0555e-05                 170.33       
      1.435e+06        -0.00017431            3.4818e-05                   -75.004               -0.00017431            3.4818e-05                  168.7       
       1.58e+06        -0.00016598            0.00011951                   -73.785               -0.00016598            0.00011951                 144.25       
      1.725e+06        -0.00016952            0.00010687                   -73.962               -0.00016952            0.00010687                 147.77       
       1.87e+06        -0.00018365            8.9397e-05                   -73.796               -0.00018365            8.9397e-05                 154.04       
      2.015e+06        -0.00019775            0.00014375                   -72.235               -0.00019775            0.00014375                 143.98       
       2.16e+06         -0.0002316            0.00015771                   -71.051                -0.0002316            0.00015771                 145.75       
      2.305e+06        -0.00028334            0.00019373                   -69.288               -0.00028334            0.00019373                 145.64       
       2.45e+06        -0.00034795            0.00022872                    -67.61               -0.00034795            0.00022872                 146.68       
      2.595e+06        -0.00054121            0.00035141                   -63.805               -0.00054121            0.00035141                    147       
       2.74e+06        -0.00080171            0.00077529                   -59.052               -0.00080171            0.00077529                 135.96       
      2.885e+06        -0.00070651             0.0014188                       -56               -0.00070651             0.0014188                 116.47       
       3.03e+06        -0.00030082             0.0020212                   -53.793               -0.00030082             0.0020212                 98.465       
      3.175e+06          0.0001996             0.0026894                   -51.383                 0.0001996             0.0026894                 85.755       

[Ahmed Ghouma]

thank you for your help but i get this error " Error using readtable (line 223) Invalid parameter name: Decimal." i am using matlab 2019b

[the cyclist]

@Stephen Cobeldick, thanks for pointing that out! I actually searched this forum and elsewhere before making my statement, but I still thought there was a high probability that someone would swoop in and correct that!

@Ahmed Ghouma, according to this comment, it is a "fairly recent" change to readtable that allows specifying the decimal separator. That was written in August 2020, so it is possible it is not in R2019b. I have R2020a on a machine, and it works there.

[Ahmed Ghouma]

@Stephen Cobeldick @the cyclist thank you very much for your help

[Ahmed Ghouma]

@Stephen Cobeldick and @the cyclist excuse me i insatalled matlab 2021a and i wanted to convert it into double to do mathematical operations so i run this function

    T = readtable('73_4.csv', 'Delimiter',';', 'DecimalSeparator',',')
    x=T(:,1);
    for k=1:numel(x)
    c(k,:)=str2double(x(k,:));
    end

but c gives a row of NaN. Could you please suggest a solution ?

[Stephen23]

"Could you please suggest a solution ?"

Umm... the obvious solution is to stop using STR2DOUBLE on numeric data.

Why are you trying to use STR2DOUBLE to convert numeric data into numeric data?

[the cyclist]

Assuming you are using the original file, with commas as separators (and not the file I posted, where I converted those to decimal points), then the syntax that @Stephen Cobeldick posted will read everything in as numeric. You don't need to convert to double anymore, because there should be strings. You can convert from the table to a numeric array if you need to:

% Read from the original file you posted. (You should keep reading from your local file.)
T = readtable('https://www.mathworks.com/matlabcentral/answers/uploaded_files/664395/73_4.csv', 'Delimiter',';', 'DecimalSeparator',',');

Warning: Column headers from the file were modified to make them valid MATLAB identifiers before creating variable names for the table. The original column headers are saved in the VariableDescriptions property.
Set 'VariableNamingRule' to 'preserve' to use the original column headers as table variable names.

c = table2array(T(:,1))

c = 201×1

1000000 1145000 1290000 1435000 1580000 1725000 1870000 2015000 2160000 2305000

[Ahmed Ghouma]

the first row is double but the others no. So y=T(:,2) is a table that s why i want to convert it into double

[the cyclist]

As I said, the syntax is for the file that has commas. You are obviously using it on the file that uses periods. (It's easy to get confused. I used the wrong file in my last post.)

USE YOUR LOCAL FILE THAT HAS COMMAS AND NOT PERIODS.

% Read from the original file you posted. (You should keep reading from your local file.)
T = readtable('https://www.mathworks.com/matlabcentral/answers/uploaded_files/664360/73_4.csv', 'Delimiter',';', 'DecimalSeparator',',');

Warning: Column headers from the file were modified to make them valid MATLAB identifiers before creating variable names for the table. The original column headers are saved in the VariableDescriptions property.
Set 'VariableNamingRule' to 'preserve' to use the original column headers as table variable names.

T(1:3,:)

ans = 3×7 table
    Frequency_Hz_    Trace1_Gain_Real    Trace1_Gain_Imaginary    Trace1_Gain_Magnitude_dB_    Trace2_Gain_Real    Trace2_Gain_Imaginary    Trace2_Gain_Phase___
    _____________    ________________    _____________________    _________________________    ________________    _____________________    ____________________

          1e+06        -0.00011834            2.7736e-06                   -78.535               -0.00011834            2.7736e-06                 178.66       
      1.145e+06        -0.00012968            3.4946e-05                   -77.438               -0.00012968            3.4946e-05                 164.92       
       1.29e+06        -0.00017933            3.0555e-05                   -74.803               -0.00017933            3.0555e-05                 170.33       

[the cyclist]

If you do use the table that has been converted, then you do not need to specify the decimal separator:

T = readtable('https://www.mathworks.com/matlabcentral/answers/uploaded_files/664395/73_4.csv','Delimiter',';');

Warning: Column headers from the file were modified to make them valid MATLAB identifiers before creating variable names for the table. The original column headers are saved in the VariableDescriptions property.
Set 'VariableNamingRule' to 'preserve' to use the original column headers as table variable names.

T(1:3,:)

ans = 3×7 table
    Frequency_Hz_    Trace1_Gain_Real    Trace1_Gain_Imaginary    Trace1_Gain_Magnitude_dB_    Trace2_Gain_Real    Trace2_Gain_Imaginary    Trace2_Gain_Phase___
    _____________    ________________    _____________________    _________________________    ________________    _____________________    ____________________

          1e+06        -0.00011834            2.7736e-06                   -78.535               -0.00011834            2.7736e-06                 178.66       
      1.145e+06        -0.00012968            3.4946e-05                   -77.438               -0.00012968            3.4946e-05                 164.92       
       1.29e+06        -0.00017933            3.0555e-05                   -74.803               -0.00017933            3.0555e-05                 170.33       

[Ahmed Ghouma]

i m not using the converted table that s why @Stephen Cobeldick suggested because i have 600 Table like that

[Ahmed Ghouma]

for the first row T(:,1) it works but for others like T(:,2)

c=table2array(T(:,2))

c is now a cell

[the cyclist]

In this code, I am using the original file you posted. It has commas, and the code gives a double when converted from a table. I don't understand what you are doing. But if you look at the screenshot you posted above, you are clearly pulling from a file that uses the decimal point.

% Read from the original file you posted. (You should keep reading from your local file.)
T = readtable('https://www.mathworks.com/matlabcentral/answers/uploaded_files/664360/73_4.csv', 'Delimiter',';', 'DecimalSeparator',',');

Warning: Column headers from the file were modified to make them valid MATLAB identifiers before creating variable names for the table. The original column headers are saved in the VariableDescriptions property.
Set 'VariableNamingRule' to 'preserve' to use the original column headers as table variable names.

c = table2array(T(:,2))

c = 201×1

1.0e+00 * -0.0001 -0.0001 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0003

class(c)

ans = 'double'

[Ahmed Ghouma]

it s really weird because i did the same thing that you did and then i had as a result ans= 'table'

[Ahmed Ghouma]

@Stephen Cobeldick @the cyclist could you please try it with this file

i think in some files it s used commas as the decimal separator, rather than a period ("decimal point") and in some others a decimal point. So have you please an idea how to specify if it is a decimal point used so i can use this

 T = readtable('73_4.csv', 'Delimiter',';')
 

and if a comma than

T = readtable('73_4.csv', 'Delimiter',';', 'DecimalSeparator',',')

i m very thankful

[Stephen23]

"i think in some files it s used commas as the decimal separator, rather than a period ("decimal point") and in some others a decimal point."

I cannot imagine any application or tool randomly alternating between decimal commas and decimal points.

What is more likely is that someone is opening and saving some of the files using Excel, which completely obliterates the original file format. If you want reliable, robust file parsing, avoid saving with MS Excel.

For looking at any text file, including CSV/TSV/SCSV files you should be using a reliable text editor, e.g. notepad++.

In the unlikely event that you really are using an application that really does create files with random decimal separators, you should first try calling

https://www.mathworks.com/help/matlab/ref/matlab.io.text.delimitedtextimportoptions.html

with the delimiter option set to ';' and see if it can correctly identify the decimal separator character.

For example you could simply call delimitedTextImportOptions twice (once with dot, once with comma) and check which one has the desired variable class (double).

If that does not work, you can simply read in a line or two of the file as text and perform your own simple detection algorithm, it would only take a few lines of code.