Error with data processing

I get an error if I run this matlab code of processing data for Discrete Wavelet Transformation analysis

[Jonas]

please post your error message and supply all required data files used in your code

[John Amoo-Otoo]

Please attached are the files

[John Amoo-Otoo]

Attached the file

[John Amoo-Otoo]

Attached the data

Answer 1

[Walter Roberson]

You need getmswtfeat from https://www.mathworks.com/matlabcentral/fileexchange/33146-feature-extraction-using-multisignal-wavelet-packet-decomposition

[John Amoo-Otoo]

That is what I have but if I run my script. Attached is the error I get

[John Amoo-Otoo]

Attached is my get feature script

[Walter Roberson]

you appear to load() a file, and use parts of some variables from the file. However we are not told what datatype is in the file.

Based on the error messages we can tell that you are expecting that the mat file contains double precision data, but that instead it has a table() object.

You need to extract data from the table object, or else you need to change the mat file to have double precision data instead of table.

It is as-if you used readtable() to read the data when readmatrix() might possibly have been a better choice. But that depends on whether the file is all numeric or not

[John Amoo-Otoo]

Please attached the data in table format

temp = load('John_FaultData0km0ohm.mat');
fieldnames(temp)

ans = 2×1 cell array
    {'DataPG0km0hmsFaultData'   }
    {'SteadyStateNoneFaultState'}

temp.DataPG0km0hmsFaultData(1:5,:)

ans = 5×2 table
     Time     DataPG0km0hmsFaultData
    ______    ______________________

      19.8          0.00030982      
    19.801          0.00030954      
    19.802          0.00030947      
    19.803          0.00031002      
    19.804          0.00030967      

temp.SteadyStateNoneFaultState(1:5,:)

ans = 5×2 table
     Time      SteadyStateNoneFaultState
    _______    _________________________

          0                    0        
    0.00099           6.4183e-09        
    0.00198           2.0619e-07        
    0.00297           1.0102e-06        
    0.00396           2.7701e-06        

[Walter Roberson]

I do not know what you want to do with the features extracted.

Note that there is a theoretical fault in the code. You remove data that has nan, but you do not take into account that that leaves a hole in the timing. You never use the timing information.

In practice the only nan is at the very last entry in the SteadyState, so removing it makes no timing difference for this data.

%% load the data first
load('John_FaultData0km0ohm.mat')

%% does this signal has any NaNs, if so remove
SteadyStateNoneFaultState = rmmissing(SteadyStateNoneFaultState);
DataPG0km0hmsFaultData = rmmissing(DataPG0km0hmsFaultData);

SSTime = SteadyStateNoneFaultState.Time;
SSNFS = SteadyStateNoneFaultState.SteadyStateNoneFaultState;

DPTime = DataPG0km0hmsFaultData.Time;
DPFD = DataPG0km0hmsFaultData.DataPG0km0hmsFaultData;

%% Define the filtering and inspect 
n                                = 8; 		%% defines window length
w                                = [-ones(n,1); ones(n,1)];
SSNFS = filter(w, n, SSNFS);
DPFD = filter(w, n, DPFD);

%% Normalize signals
med_training                     = prctile(SSNFS,50);
iqr_training                     = iqr(SSNFS);
SSNFS                            = (SSNFS-med_training)./iqr_training;

med_fault                        = prctile(DPFD,50);
iqr_fault                        = iqr(DPFD);
DPFD                             = (DPFD-med_fault)./iqr_fault;

% Plot & Observe the data
subplot(2,1,1)
plot(DPTime, DPFD)
title('filtered DataPG0km0hmsFaultData')

subplot(2,1,2)
plot(SSTime, SSNFS)
title('filtered SteadyStateNoneFaultState')

%% Let's observe the FFT power spectrum for differences
feat_fault = getmswtfeat(DPFD,32,16,100000);
feat_Good = getmswtfeat(SSNFS,32,16,100000);

[John Amoo-Otoo]

Walter, thank you so much for debugging

[John Amoo-Otoo]

Walter, could you also debug the errors on this script.

[John Amoo-Otoo]

Walter you are a Guru and super consultant in MATLAB. the script in data proicessing runs perfectly. The only script now that requires debugging is getmsfeat.m. After that I will be squared away

[John Amoo-Otoo]

The getsmsfeat.m is the Discrete Wavelet Transform Decomposition script using different mother wavelets( Daubechies, Symlets, Coiflets,etc)

[Walter Roberson]

What error are you seeing? Earlier when I ran your code after fixing the other problems, the only problem that I had with the feature extraction that you posted earlier, was that the code needed Wavelet Toolbox. When I installed that, it ran to completion. If it runs without error but does not produce the output you want then you will need to be specific about what I should be looking for.

[John Amoo-Otoo]

I will see if I can re-install my Wavelet ToolBox

[John Amoo-Otoo]

Walter, please attached is the error I have getsfeature

[John Amoo-Otoo]

I have attached the script again

[Walter Roberson]

MATLAB would have stopped after a single error, not continued to tell you about all the different problems encountered in getmswtfeat .

I do not have any problems on my system when I run the processdata that I posted.

Which MATLAB release are you using?

That error you are getting about nargin would only apply if somehow your getmswtfeat is a script instead of a function. What you attached is a function. You should

which -all getmswtfeat

to check in case somehow you have an additional getmswtfeat that is interfering.

[John Amoo-Otoo]

Walter, Good day, from the attached, for an 8 level deocmposition for daubechies 2, I am expecting 9 features from column 1 through column 9. In the columns are these the detailed coefficients from column 1 through 8 and the column 9 should be the approximate. Are these entropies or detailed coefficients?

[Walter Roberson]

@John Amoo-Otoo

Those are entropies.

feature_out = zeros(numwin,(J+1)*Nsignals);

        tab_entropy(:,k)     = -sum(prob.*log(prob),2);%./size(percentENER(:,st:en),2);

    feature_out(:,(1:((J+1)))+(dims-1)*(J+1)) =tab_entropy;

feature_out is not written to anywhere else, and tab_entropy is not written to anywhere else (other than pre-allocation with zeros)

[John Amoo-Otoo]

Walter, it looks like the column one is the detailed coefficients and the plots is the entropy. Just checking if that is true. Also if I am looking for the highest detailed coefficients I can see negative and positive number the column, which one is the highest.

[Walter Roberson]

Line 51: feature_out = zeros(numwin,(J+1)*Nsignals);

Line 97: tab_entropy = zeros(numOfSIGs,level+1);

Line 110: tab_entropy(:,k) = -sum(prob.*log(prob),2);%./size(percentENER(:,st:en),2);

Line 114: feature_out(:,(1:((J+1)))+(dims-1)*(J+1)) =tab_entropy;

Those are the only lines that write to feature_out or tab_entropy that are not commented out. You can see that column 1 of tab_entropy is not written to differently than the other columns, and you can see that nothing other than tab_entropy is written into feature_out (other than the initial zeros.)

Therefore, column 1 of the output is an entropy, not a detailed coefficient.

[John Amoo-Otoo]

Walt, please could you look into this attached error. My last simulation run perfectly and I changed the data to process another data and feature fault was given me an error.

Also for the feature out code if I want to add a script for tab detailed coefficients juts like tab entropy how do I do that

[John Amoo-Otoo]

Walter, I was able to resolve the error. could you send me a script line for tab calculating Detailed Coefficients

[Walter Roberson]

I do not know how to calculate detailed coefficients.

[John Amoo-Otoo]

Also if you have a script for tabing Detailed coefficient you can send it to me

[Walter Roberson]

As I do not have any script for that purpose, should I be arranging to send you the lack of script, or should I be arranging to not send you the script that I do not have?

[John Amoo-Otoo]

Walter, please could you look into this error. The script is for calculating the detailed coefficients using Matlab Discrete Wavelet Transform

[Walter Roberson]

plot(DataPG0km5hmsFaultData, 'Time', 'DataPG0km5hmsFaultData')

[John Amoo-Otoo]

walter, please could you look into gthis error for me

[Walter Roberson]

As I reported to you in https://www.mathworks.com/matlabcentral/answers/1763300-error-with-data-processing#comment_2278845 your very last entry in your data is NaN. wavedec() cannot handle NaN, so you need to filter your input before calling wavedec() ... like I showed in that comment.

[John Amoo-Otoo]

Walter, Thank you it worked. I have been able to add a single line script that will remove the NANs

[John Amoo-Otoo]

Walt, a quick question. I am processing another script with a signal that I added Gaussian Noise to mimic the intrusion of lightning and switchig disturbance during the fault condition signal. and attached is the error I got. Please could you look into it at your convenience

[Walter Roberson]

The code before the error line does not define any variable named signal but it does define a variable named signals

[John Amoo-Otoo]

Thanks walt

Answer 2

[John Amoo-Otoo]

Jonas, please attached is my error for data pocessing

[John Amoo-Otoo]

Attached is my data processing code

[John Amoo-Otoo]

Jonas, please could you look into this error for me